Jul 20 16:43:23.968095 ip-10-0-130-18 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 20 16:43:23.968107 ip-10-0-130-18 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 20 16:43:23.968114 ip-10-0-130-18 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 20 16:43:23.968407 ip-10-0-130-18 systemd[1]: Failed to start Kubernetes Kubelet. Jul 20 16:43:34.159909 ip-10-0-130-18 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 20 16:43:34.159929 ip-10-0-130-18 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 55ba1f56dcfe4b87a3ce6755d3a3703c -- Jul 20 16:45:35.086099 ip-10-0-130-18 systemd[1]: Starting Kubernetes Kubelet... Jul 20 16:45:35.529188 ip-10-0-130-18 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 16:45:35.529188 ip-10-0-130-18 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 20 16:45:35.529188 ip-10-0-130-18 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 16:45:35.529188 ip-10-0-130-18 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 20 16:45:35.529188 ip-10-0-130-18 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 16:45:35.530513 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.530424 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 20 16:45:35.535012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.534993 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 16:45:35.535012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535012 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535015 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535019 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535022 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535026 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535029 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535032 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535034 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535037 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535040 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535042 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535045 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535048 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535056 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535059 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535061 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535064 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535066 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535069 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535071 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 16:45:35.535083 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535074 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535077 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535079 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535082 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535085 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535087 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535090 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535095 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535099 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535102 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535105 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535107 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535110 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535113 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535116 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535118 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535121 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535124 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535126 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 16:45:35.535556 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535129 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535133 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535136 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535139 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535141 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535144 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535147 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535150 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535152 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535155 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535158 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535160 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535163 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535166 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535169 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535173 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535176 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535179 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535182 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535184 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 16:45:35.536026 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535187 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535190 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535192 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535195 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535197 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535199 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535202 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535204 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535207 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535209 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535212 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535215 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535217 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535253 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535257 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535260 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535263 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535266 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535268 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535271 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 16:45:35.536512 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535274 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535276 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535279 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535282 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535285 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535287 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535292 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535295 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535298 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535302 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535305 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535307 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535311 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535315 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535317 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535320 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535323 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535326 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.535328 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536057 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 16:45:35.536999 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536063 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536066 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536070 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536072 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536076 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536079 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536081 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536084 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536087 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536089 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536092 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536095 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536097 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536100 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536104 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536108 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536111 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536114 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536117 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 16:45:35.537467 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536120 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536122 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536140 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536143 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536146 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536149 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536152 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536155 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536158 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536161 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536163 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536166 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536168 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536172 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536176 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536180 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536182 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536185 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536195 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 16:45:35.538001 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536198 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536200 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536203 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536205 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536207 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536210 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536212 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536215 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536218 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536225 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536228 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536231 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536233 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536237 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536239 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536242 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536245 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536247 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536250 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536253 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 16:45:35.538540 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536255 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536257 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536260 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536262 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536265 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536267 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536270 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536272 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536275 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536277 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536280 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536282 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536285 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536287 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536290 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536292 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536295 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536297 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536300 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536302 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 16:45:35.539045 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536304 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536306 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536309 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536312 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536314 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536317 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536319 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536322 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536324 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536327 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536329 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536332 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536335 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536338 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536340 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536343 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536346 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536348 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536350 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536353 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 16:45:35.539524 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536714 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536721 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536725 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536728 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536731 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536734 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536737 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536740 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536745 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536749 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536752 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536754 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536757 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536760 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536763 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536765 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536768 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536787 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536790 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 16:45:35.540037 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536792 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536795 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536798 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536800 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536803 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536806 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536808 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536811 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536814 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536816 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536819 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536821 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536824 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536826 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536829 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536832 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536834 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536837 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536840 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536843 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 16:45:35.540511 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536846 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536848 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536851 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536853 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536856 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536858 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536861 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536864 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536868 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536871 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536874 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536877 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536880 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536882 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536885 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536887 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536890 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536892 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536895 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 16:45:35.541012 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536897 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536899 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536902 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536904 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536907 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536909 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536911 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536914 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536917 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536919 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536922 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536924 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536928 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536931 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536934 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536936 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536939 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536941 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536943 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536946 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 16:45:35.541474 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536949 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536951 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536955 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536957 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536960 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536962 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536965 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536968 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536970 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536973 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536975 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536978 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536980 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536983 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536985 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536988 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536990 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536993 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536995 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.536998 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 16:45:35.541976 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537000 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537089 2566 flags.go:64] FLAG: --address="0.0.0.0" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537100 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537110 2566 flags.go:64] FLAG: --anonymous-auth="true" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537115 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537121 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537124 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537128 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537133 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537136 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537139 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537143 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537146 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537149 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537152 2566 flags.go:64] FLAG: --cgroup-root="" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537155 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537158 2566 flags.go:64] FLAG: --client-ca-file="" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537161 2566 flags.go:64] FLAG: --cloud-provider="external" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537163 2566 flags.go:64] FLAG: --cluster-dns="[]" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537168 2566 flags.go:64] FLAG: --cluster-domain="" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537170 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537173 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537176 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537180 2566 flags.go:64] FLAG: --container-log-max-files="5" Jul 20 16:45:35.542467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537183 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537186 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537189 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537192 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537195 2566 flags.go:64] FLAG: --contention-profiling="false" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537198 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537200 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537204 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537206 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537211 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537214 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537217 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537222 2566 flags.go:64] FLAG: --enable-load-reader="false" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537226 2566 flags.go:64] FLAG: --enable-server="true" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537229 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537235 2566 flags.go:64] FLAG: --event-burst="100" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537239 2566 flags.go:64] FLAG: --event-qps="50" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537242 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537244 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537247 2566 flags.go:64] FLAG: --eviction-hard="" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537251 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537254 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537257 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537260 2566 flags.go:64] FLAG: --eviction-soft="" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537263 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 20 16:45:35.543082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537265 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537268 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537271 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537274 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537277 2566 flags.go:64] FLAG: --fail-swap-on="true" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537279 2566 flags.go:64] FLAG: --feature-gates="" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537283 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537286 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537289 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537292 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537295 2566 flags.go:64] FLAG: --healthz-port="10248" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537298 2566 flags.go:64] FLAG: --help="false" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537300 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-130-18.ec2.internal" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537304 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537307 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537310 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537313 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537316 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537319 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537323 2566 flags.go:64] FLAG: --image-service-endpoint="" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537326 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537328 2566 flags.go:64] FLAG: --kube-api-burst="100" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537333 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537336 2566 flags.go:64] FLAG: --kube-api-qps="50" Jul 20 16:45:35.543693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537339 2566 flags.go:64] FLAG: --kube-reserved="" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537342 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537345 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537348 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537350 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537353 2566 flags.go:64] FLAG: --lock-file="" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537356 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537359 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537361 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537366 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537369 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537372 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537375 2566 flags.go:64] FLAG: --logging-format="text" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537378 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537381 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537384 2566 flags.go:64] FLAG: --manifest-url="" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537386 2566 flags.go:64] FLAG: --manifest-url-header="" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537391 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537393 2566 flags.go:64] FLAG: --max-open-files="1000000" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537397 2566 flags.go:64] FLAG: --max-pods="110" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537400 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537403 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537406 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537409 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537412 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 20 16:45:35.544343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537415 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537418 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537426 2566 flags.go:64] FLAG: --node-status-max-images="50" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537429 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537432 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537436 2566 flags.go:64] FLAG: --pod-cidr="" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537440 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537446 2566 flags.go:64] FLAG: --pod-manifest-path="" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537448 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537451 2566 flags.go:64] FLAG: --pods-per-core="0" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537454 2566 flags.go:64] FLAG: --port="10250" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537457 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537460 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f3e467268bf26fcf" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537463 2566 flags.go:64] FLAG: --qos-reserved="" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537466 2566 flags.go:64] FLAG: --read-only-port="10255" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537469 2566 flags.go:64] FLAG: --register-node="true" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537471 2566 flags.go:64] FLAG: --register-with-taints="" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537475 2566 flags.go:64] FLAG: --registry-burst="10" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537478 2566 flags.go:64] FLAG: --registry-qps="5" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537480 2566 flags.go:64] FLAG: --reserved-cpus="" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537483 2566 flags.go:64] FLAG: --reserved-memory="" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537487 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537490 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537493 2566 flags.go:64] FLAG: --rotate-certificates="false" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537495 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 20 16:45:35.545400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537498 2566 flags.go:64] FLAG: --runonce="false" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537501 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537503 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537506 2566 flags.go:64] FLAG: --seccomp-default="false" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537509 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537512 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537514 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537517 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537520 2566 flags.go:64] FLAG: --storage-driver-password="root" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537524 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537527 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537529 2566 flags.go:64] FLAG: --storage-driver-user="root" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537532 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537536 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537540 2566 flags.go:64] FLAG: --system-cgroups="" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537543 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537548 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537551 2566 flags.go:64] FLAG: --tls-cert-file="" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537554 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537558 2566 flags.go:64] FLAG: --tls-min-version="" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537561 2566 flags.go:64] FLAG: --tls-private-key-file="" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537564 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537566 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537569 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537572 2566 flags.go:64] FLAG: --v="2" Jul 20 16:45:35.546452 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537576 2566 flags.go:64] FLAG: --version="false" Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537589 2566 flags.go:64] FLAG: --vmodule="" Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537593 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.537596 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537713 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537716 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537720 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537723 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537726 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537730 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537733 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537735 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537738 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537740 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537743 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537746 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537750 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537753 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537758 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 16:45:35.547139 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537761 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537767 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537783 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537787 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537790 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537792 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537795 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537808 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537810 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537813 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537816 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537818 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537821 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537823 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537826 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537828 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537831 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537833 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537836 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537838 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 16:45:35.547827 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537841 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537843 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537846 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537848 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537851 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537853 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537856 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537858 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537860 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537863 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537867 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537870 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537872 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537877 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537880 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537882 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537885 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537888 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537890 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537893 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 16:45:35.548682 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537895 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537898 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537900 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537903 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537905 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537909 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537912 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537915 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537951 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537959 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537962 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537969 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537972 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537975 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537979 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537982 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537985 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537988 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537991 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537993 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 16:45:35.549363 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537996 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.537999 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538002 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538005 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538008 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538011 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538013 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538016 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538019 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538022 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538024 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538026 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538030 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538033 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538036 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538038 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538040 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538043 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538046 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538049 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 16:45:35.550118 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538054 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 16:45:35.550964 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538058 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 16:45:35.550964 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538061 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 16:45:35.550964 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.538063 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 16:45:35.550964 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.538737 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 16:45:35.550964 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.546550 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 20 16:45:35.550964 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.546574 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 20 16:45:35.550964 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.547446 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 20 16:45:35.550964 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.547460 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 20 16:45:35.550964 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547578 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 16:45:35.550964 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547586 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 16:45:35.550964 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547592 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 16:45:35.550964 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547597 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 16:45:35.550964 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547602 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 16:45:35.550964 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547607 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 16:45:35.550964 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547612 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547617 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547622 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547626 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547631 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547636 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547641 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547645 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547650 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547655 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547659 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547663 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547667 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547672 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547676 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547680 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547685 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547689 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547693 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547697 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 16:45:35.551385 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547702 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547706 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547710 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547716 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547722 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547726 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547730 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547735 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547739 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547743 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547747 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547752 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547755 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547760 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547764 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547793 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547798 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547803 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547807 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 16:45:35.552027 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547812 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547816 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547820 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547824 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547828 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547832 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547838 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547843 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547848 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547853 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547857 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547861 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547865 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547869 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547873 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547878 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547883 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547889 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547893 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 20 16:45:35.552520 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547897 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547902 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547906 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547910 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547914 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547918 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547922 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547926 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547930 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547934 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547939 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547943 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547947 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547951 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547957 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547964 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547968 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547972 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547976 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 16:45:35.553004 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547981 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 16:45:35.553526 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547985 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 16:45:35.553526 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547989 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 16:45:35.553526 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547993 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 16:45:35.553526 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.547997 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 16:45:35.553526 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548002 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 16:45:35.553526 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548006 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 16:45:35.553526 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548010 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 16:45:35.553526 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548014 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 16:45:35.553526 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548018 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 16:45:35.553526 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548023 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 16:45:35.553526 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548029 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 16:45:35.553526 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548036 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 16:45:35.553526 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548041 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 16:45:35.553526 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548045 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 16:45:35.553526 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548050 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 16:45:35.554059 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.548059 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 16:45:35.554059 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548280 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 16:45:35.554059 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548290 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 16:45:35.554059 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548296 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 16:45:35.554059 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548300 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 16:45:35.554059 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548304 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 16:45:35.554059 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548308 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 16:45:35.554059 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548312 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 16:45:35.554059 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548317 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 16:45:35.554059 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548321 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 16:45:35.554059 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548325 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 16:45:35.554059 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548329 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 16:45:35.554059 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548333 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 16:45:35.554059 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548337 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 16:45:35.554059 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548342 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548346 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548350 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548355 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548359 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548363 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548368 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548372 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548377 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548381 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548385 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548389 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548393 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548398 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548402 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548411 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548418 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548422 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548427 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 16:45:35.554478 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548431 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548435 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548439 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548443 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548447 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548451 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548455 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548460 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548464 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548468 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548472 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548476 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548480 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548484 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548488 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548492 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548497 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548501 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548505 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548510 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 16:45:35.555006 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548513 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548520 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548526 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548531 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548536 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548540 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548545 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548550 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548554 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548559 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548564 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548569 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548573 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548577 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548581 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548585 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548589 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548593 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548597 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 16:45:35.555486 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548601 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548605 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548609 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548613 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548617 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548622 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548626 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548630 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548634 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548639 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548643 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548650 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548655 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548661 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548666 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548670 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548674 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548678 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548682 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548686 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 16:45:35.556072 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548690 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 16:45:35.556564 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548694 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 16:45:35.556564 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548698 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 16:45:35.556564 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548704 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 16:45:35.556564 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548708 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 16:45:35.556564 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548712 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 16:45:35.556564 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548717 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 16:45:35.556564 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:35.548721 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 16:45:35.556564 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.548728 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 16:45:35.556564 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.548950 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jul 20 16:45:35.556564 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.551610 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 20 16:45:35.556564 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.553276 2566 server.go:1019] "Starting client certificate rotation" Jul 20 16:45:35.556564 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.553376 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 16:45:35.556564 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.553410 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 16:45:35.557191 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.557172 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 16:45:35.564473 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.564444 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 16:45:35.573341 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.573321 2566 log.go:25] "Validated CRI v1 runtime API" Jul 20 16:45:35.575643 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.575629 2566 log.go:25] "Validated CRI v1 image API" Jul 20 16:45:35.577423 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.577407 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 20 16:45:35.579918 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.579896 2566 fs.go:135] Filesystem UUIDs: map[5371f89b-14d1-4e85-9a01-b3bf55003707:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 960ce19e-5178-4dfd-a720-fe6a1b6c696e:/dev/nvme0n1p4] Jul 20 16:45:35.579971 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.579917 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 20 16:45:35.581680 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.581659 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 16:45:35.585488 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.585368 2566 manager.go:216] Machine: {Timestamp:2026-07-20 16:45:35.583581654 +0000 UTC m=+0.369580725 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099663 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24320f5a765ab141b7308287131d7b SystemUUID:ec24320f-5a76-5ab1-41b7-308287131d7b BootID:55ba1f56-dcfe-4b87-a3ce-6755d3a3703c Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:49:87:db:dd:79 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:49:87:db:dd:79 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5e:18:4c:30:a4:7a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 20 16:45:35.585488 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.585479 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 20 16:45:35.585639 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.585564 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 20 16:45:35.589124 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.589094 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 20 16:45:35.589275 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.589127 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-18.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 20 16:45:35.589874 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.589864 2566 topology_manager.go:138] "Creating topology manager with none policy" Jul 20 16:45:35.589913 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.589877 2566 container_manager_linux.go:309] "Creating device plugin manager" Jul 20 16:45:35.589913 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.589893 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 16:45:35.590583 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.590573 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 16:45:35.591867 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.591857 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 20 16:45:35.595199 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.595185 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 20 16:45:35.595315 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.595306 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 20 16:45:35.595359 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.595337 2566 kubelet.go:486] "Attempting to sync node with API server" Jul 20 16:45:35.595359 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.595350 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 20 16:45:35.595427 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.595362 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 20 16:45:35.595427 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.595373 2566 kubelet.go:398] "Adding apiserver pod source" Jul 20 16:45:35.595427 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.595383 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 20 16:45:35.595846 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.595829 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 16:45:35.595912 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.595858 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 16:45:35.598891 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.598848 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 20 16:45:35.601955 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.601937 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 20 16:45:35.602040 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.601971 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 20 16:45:35.603476 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.603464 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 20 16:45:35.603521 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.603486 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 20 16:45:35.603521 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.603497 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 20 16:45:35.603521 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.603503 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 20 16:45:35.603521 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.603512 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 20 16:45:35.603521 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.603517 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 20 16:45:35.603668 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.603525 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 20 16:45:35.603668 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.603530 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 20 16:45:35.603668 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.603537 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 20 16:45:35.603668 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.603543 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 20 16:45:35.603668 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.603551 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 20 16:45:35.603668 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.603557 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 20 16:45:35.604362 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.604351 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 20 16:45:35.604394 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.604363 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 20 16:45:35.607835 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.607795 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-18.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 20 16:45:35.608006 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.607995 2566 server.go:1268] "Started kubelet" Jul 20 16:45:35.608100 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.608071 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 20 16:45:35.608262 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.608214 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 20 16:45:35.608342 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.608283 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 20 16:45:35.608484 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:35.608410 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 20 16:45:35.608484 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:35.608465 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-18.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 20 16:45:35.608891 ip-10-0-130-18 systemd[1]: Started Kubernetes Kubelet. Jul 20 16:45:35.610169 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.610155 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 20 16:45:35.610230 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.610211 2566 server.go:310] "Adding debug handlers to kubelet server" Jul 20 16:45:35.611877 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.611848 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t7s4m" Jul 20 16:45:35.613932 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:35.613902 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 20 16:45:35.614329 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:35.613325 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-18.ec2.internal.18c40d436e2ce744 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-18.ec2.internal,UID:ip-10-0-130-18.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-18.ec2.internal,},FirstTimestamp:2026-07-20 16:45:35.607965508 +0000 UTC m=+0.393964578,LastTimestamp:2026-07-20 16:45:35.607965508 +0000 UTC m=+0.393964578,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-18.ec2.internal,}" Jul 20 16:45:35.617173 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.617157 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t7s4m" Jul 20 16:45:35.617911 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.617898 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 20 16:45:35.617977 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.617962 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 20 16:45:35.618045 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:35.618006 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-18.ec2.internal\" not found" Jul 20 16:45:35.618530 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.618494 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 20 16:45:35.618530 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.618513 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 20 16:45:35.618741 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.618491 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 20 16:45:35.619032 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.619006 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 20 16:45:35.619122 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.619036 2566 factory.go:55] Registering systemd factory Jul 20 16:45:35.619122 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.619049 2566 factory.go:223] Registration of the systemd container factory successfully Jul 20 16:45:35.619202 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.619152 2566 reconstruct.go:97] "Volume reconstruction finished" Jul 20 16:45:35.619202 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.619162 2566 reconciler.go:29] "Reconciler: start to sync state" Jul 20 16:45:35.619309 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.619292 2566 factory.go:153] Registering CRI-O factory Jul 20 16:45:35.619309 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.619312 2566 factory.go:223] Registration of the crio container factory successfully Jul 20 16:45:35.619444 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.619333 2566 factory.go:103] Registering Raw factory Jul 20 16:45:35.619444 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.619436 2566 manager.go:1196] Started watching for new ooms in manager Jul 20 16:45:35.620141 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.620126 2566 manager.go:349] Starting recovery of all containers Jul 20 16:45:35.622648 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:35.622619 2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-18.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 20 16:45:35.622808 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:35.622763 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 20 16:45:35.632247 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.632220 2566 manager.go:354] Recovery completed Jul 20 16:45:35.633575 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:35.633551 2566 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 20 16:45:35.636685 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.636667 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:45:35.638572 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.638557 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:45:35.638643 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.638589 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:45:35.638643 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.638599 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:45:35.639132 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.639122 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 20 16:45:35.639132 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.639130 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 20 16:45:35.639229 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.639146 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 20 16:45:35.639229 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.639212 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:45:35.640305 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.640294 2566 policy_none.go:49] "None policy: Start" Jul 20 16:45:35.640338 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.640310 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 20 16:45:35.640338 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.640319 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 20 16:45:35.641743 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.641732 2566 policy_none.go:47] "Start" Jul 20 16:45:35.648138 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.648112 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:45:35.648217 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.648145 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:45:35.648217 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.648158 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:45:35.674401 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.674384 2566 manager.go:337] "Starting Device Plugin manager" Jul 20 16:45:35.674502 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:35.674422 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 20 16:45:35.674502 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.674437 2566 server.go:87] "Starting device plugin registration server" Jul 20 16:45:35.674722 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.674711 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 20 16:45:35.674762 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.674724 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 20 16:45:35.674924 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.674899 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 20 16:45:35.675040 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.674987 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 20 16:45:35.675040 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.674995 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 20 16:45:35.675425 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:35.675407 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 20 16:45:35.675513 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:35.675450 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-18.ec2.internal\" not found" Jul 20 16:45:35.775022 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.774987 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:45:35.776113 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.776066 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:45:35.776113 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.776107 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:45:35.776240 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.776128 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:45:35.776240 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.776160 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-18.ec2.internal" Jul 20 16:45:35.784590 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.784513 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-18.ec2.internal" Jul 20 16:45:35.784590 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:35.784538 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-18.ec2.internal\": node \"ip-10-0-130-18.ec2.internal\" not found" Jul 20 16:45:35.797449 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:35.797407 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-18.ec2.internal\" not found" Jul 20 16:45:35.806925 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.806892 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 20 16:45:35.808316 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.808292 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 20 16:45:35.808405 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.808338 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 20 16:45:35.808405 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.808362 2566 kubelet.go:2448] "Starting kubelet main sync loop" Jul 20 16:45:35.808405 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:35.808401 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 20 16:45:35.811272 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.811250 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:45:35.898211 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:35.898162 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-18.ec2.internal\" not found" Jul 20 16:45:35.909338 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.909308 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-18.ec2.internal"] Jul 20 16:45:35.909457 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.909441 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:45:35.910511 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.910496 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:45:35.910564 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.910524 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:45:35.910564 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.910535 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:45:35.911812 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.911801 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:45:35.911930 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.911916 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal" Jul 20 16:45:35.911994 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.911949 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:45:35.912632 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.912614 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:45:35.912737 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.912646 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:45:35.912737 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.912656 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:45:35.912737 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.912613 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:45:35.912737 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.912704 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:45:35.912737 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.912717 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:45:35.913833 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.913816 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-18.ec2.internal" Jul 20 16:45:35.913940 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.913841 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:45:35.914557 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.914544 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:45:35.914625 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.914572 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:45:35.914625 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.914586 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:45:35.920954 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.920936 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/976724d148be7ca265b295555429a347-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal\" (UID: \"976724d148be7ca265b295555429a347\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal" Jul 20 16:45:35.921023 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.920961 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/976724d148be7ca265b295555429a347-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal\" (UID: \"976724d148be7ca265b295555429a347\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal" Jul 20 16:45:35.921023 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:35.920978 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e496a6e153fcce1aa86c117a388fbbcb-config\") pod \"kube-apiserver-proxy-ip-10-0-130-18.ec2.internal\" (UID: \"e496a6e153fcce1aa86c117a388fbbcb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-18.ec2.internal" Jul 20 16:45:35.936468 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:35.936449 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-18.ec2.internal\" not found" node="ip-10-0-130-18.ec2.internal" Jul 20 16:45:35.939868 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:35.939851 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-18.ec2.internal\" not found" node="ip-10-0-130-18.ec2.internal" Jul 20 16:45:35.998566 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:35.998535 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-18.ec2.internal\" not found" Jul 20 16:45:36.021947 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.021919 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/976724d148be7ca265b295555429a347-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal\" (UID: \"976724d148be7ca265b295555429a347\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal" Jul 20 16:45:36.022063 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.021950 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/976724d148be7ca265b295555429a347-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal\" (UID: \"976724d148be7ca265b295555429a347\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal" Jul 20 16:45:36.022063 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.021969 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e496a6e153fcce1aa86c117a388fbbcb-config\") pod \"kube-apiserver-proxy-ip-10-0-130-18.ec2.internal\" (UID: \"e496a6e153fcce1aa86c117a388fbbcb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-18.ec2.internal" Jul 20 16:45:36.022063 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.021999 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e496a6e153fcce1aa86c117a388fbbcb-config\") pod \"kube-apiserver-proxy-ip-10-0-130-18.ec2.internal\" (UID: \"e496a6e153fcce1aa86c117a388fbbcb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-18.ec2.internal" Jul 20 16:45:36.022063 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.022028 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/976724d148be7ca265b295555429a347-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal\" (UID: \"976724d148be7ca265b295555429a347\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal" Jul 20 16:45:36.022180 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.022027 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/976724d148be7ca265b295555429a347-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal\" (UID: \"976724d148be7ca265b295555429a347\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal" Jul 20 16:45:36.099252 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:36.099221 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-18.ec2.internal\" not found" Jul 20 16:45:36.199806 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:36.199761 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-18.ec2.internal\" not found" Jul 20 16:45:36.240273 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.240248 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal" Jul 20 16:45:36.242410 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.242390 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-18.ec2.internal" Jul 20 16:45:36.300484 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:36.300445 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-18.ec2.internal\" not found" Jul 20 16:45:36.401059 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:36.400977 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-18.ec2.internal\" not found" Jul 20 16:45:36.441967 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.441934 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:45:36.503789 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:36.503749 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-18.ec2.internal\" not found" Jul 20 16:45:36.544162 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.544133 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:45:36.553067 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.553038 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 20 16:45:36.553207 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.553174 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 16:45:36.553333 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.553221 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 16:45:36.553333 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.553223 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 16:45:36.595870 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.595835 2566 apiserver.go:52] "Watching apiserver" Jul 20 16:45:36.616499 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.615417 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 20 16:45:36.616499 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.616259 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-t2zqd","kube-system/konnectivity-agent-vsvts","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb","openshift-dns/node-resolver-88g67","openshift-image-registry/node-ca-ps2lf","openshift-multus/multus-ksj8s","openshift-multus/network-metrics-daemon-6xs67","openshift-network-operator/iptables-alerter-hxn82","openshift-cluster-node-tuning-operator/tuned-xk5jb","openshift-multus/multus-additional-cni-plugins-27ncp","openshift-network-diagnostics/network-check-target-qfqcm","openshift-ovn-kubernetes/ovnkube-node-l5n5k"] Jul 20 16:45:36.618326 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.618304 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 20 16:45:36.618382 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.618326 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-18.ec2.internal" Jul 20 16:45:36.618508 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.618490 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:36.618758 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:36.618727 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:45:36.619617 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.619597 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vsvts" Jul 20 16:45:36.620853 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.620835 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.621309 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.621288 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tv7gn\"" Jul 20 16:45:36.621492 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.621465 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 20 16:45:36.621585 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.621511 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 20 16:45:36.621963 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.621947 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-88g67" Jul 20 16:45:36.622466 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.622424 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 16:40:36 +0000 UTC" deadline="2028-02-10 07:10:57.649710137 +0000 UTC" Jul 20 16:45:36.622466 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.622465 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13670h25m21.02724839s" Jul 20 16:45:36.622769 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.622753 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 20 16:45:36.622871 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.622837 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 20 16:45:36.623007 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.622984 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-s8n4p\"" Jul 20 16:45:36.623115 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.622989 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 20 16:45:36.623335 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.623319 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ps2lf" Jul 20 16:45:36.623844 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.623826 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 20 16:45:36.623912 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.623826 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 20 16:45:36.623985 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.623970 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f75sm\"" Jul 20 16:45:36.624724 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.624710 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.625083 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.625060 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/808cf239-1e7e-484f-89f2-e8e9ad673366-kubelet-config\") pod \"global-pull-secret-syncer-t2zqd\" (UID: \"808cf239-1e7e-484f-89f2-e8e9ad673366\") " pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:36.625148 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.625097 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/808cf239-1e7e-484f-89f2-e8e9ad673366-dbus\") pod \"global-pull-secret-syncer-t2zqd\" (UID: \"808cf239-1e7e-484f-89f2-e8e9ad673366\") " pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:36.625148 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.625130 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret\") pod \"global-pull-secret-syncer-t2zqd\" (UID: \"808cf239-1e7e-484f-89f2-e8e9ad673366\") " pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:36.625229 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.625155 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e820f776-0544-43e3-a7f3-de0235215c36-agent-certs\") pod \"konnectivity-agent-vsvts\" (UID: \"e820f776-0544-43e3-a7f3-de0235215c36\") " pod="kube-system/konnectivity-agent-vsvts" Jul 20 16:45:36.625229 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.625180 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e820f776-0544-43e3-a7f3-de0235215c36-konnectivity-ca\") pod \"konnectivity-agent-vsvts\" (UID: \"e820f776-0544-43e3-a7f3-de0235215c36\") " pod="kube-system/konnectivity-agent-vsvts" Jul 20 16:45:36.625329 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.625236 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 20 16:45:36.625329 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.625251 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 20 16:45:36.625427 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.625327 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 20 16:45:36.625478 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.625469 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w2gzp\"" Jul 20 16:45:36.626019 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.626001 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:36.626095 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:36.626075 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:45:36.626611 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.626596 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 20 16:45:36.626679 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.626665 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 20 16:45:36.627204 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.627186 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hxn82" Jul 20 16:45:36.627327 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.627310 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 20 16:45:36.627433 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.627416 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 20 16:45:36.627497 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.627415 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-npdc4\"" Jul 20 16:45:36.628478 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.628462 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.628937 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.628922 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 20 16:45:36.629105 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.629088 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xhg7p\"" Jul 20 16:45:36.629277 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.629264 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 20 16:45:36.629421 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.629409 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 20 16:45:36.629595 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.629579 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 16:45:36.629668 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.629656 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal" Jul 20 16:45:36.629817 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.629801 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.630557 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.630539 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 20 16:45:36.630718 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.630705 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 20 16:45:36.630784 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.630727 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nvr7h\"" Jul 20 16:45:36.631030 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.631013 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:36.631125 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:36.631100 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:45:36.631727 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.631709 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8ghx2\"" Jul 20 16:45:36.631836 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.631757 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 20 16:45:36.632050 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.632030 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 20 16:45:36.632326 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.632309 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-18.ec2.internal"] Jul 20 16:45:36.632437 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.632423 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.633055 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.633038 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 16:45:36.635196 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.635141 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 20 16:45:36.635318 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.635212 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 20 16:45:36.635318 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.635218 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 20 16:45:36.635489 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.635402 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 20 16:45:36.635559 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.635545 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 20 16:45:36.635615 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.635545 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-j6745\"" Jul 20 16:45:36.635858 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.635829 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 20 16:45:36.641351 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.641137 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 16:45:36.641857 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.641832 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal"] Jul 20 16:45:36.660053 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.659998 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-p2swg" Jul 20 16:45:36.669492 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.669461 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-p2swg" Jul 20 16:45:36.720771 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.720594 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 20 16:45:36.726082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726059 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/808cf239-1e7e-484f-89f2-e8e9ad673366-dbus\") pod \"global-pull-secret-syncer-t2zqd\" (UID: \"808cf239-1e7e-484f-89f2-e8e9ad673366\") " pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:36.726177 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726092 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-kubelet\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.726177 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726115 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.726251 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726192 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs\") pod \"network-metrics-daemon-6xs67\" (UID: \"b18dfe54-9dfb-4d77-b2f5-5fe6837623d5\") " pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:36.726251 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726244 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-run-openvswitch\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.726314 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726254 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/808cf239-1e7e-484f-89f2-e8e9ad673366-dbus\") pod \"global-pull-secret-syncer-t2zqd\" (UID: \"808cf239-1e7e-484f-89f2-e8e9ad673366\") " pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:36.726314 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726270 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-run-ovn\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.726314 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726295 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-run-ovn-kubernetes\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.726411 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726324 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/81f36e5a-cb50-4407-820e-506a836f18dc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.726411 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726346 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-run-netns\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.726411 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726370 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-var-lib-openvswitch\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.726411 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726394 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-host-var-lib-cni-multus\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.726531 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726417 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-hostroot\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.726531 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726442 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a63896ac-798f-4830-be8d-76fdfc199e0c-host-slash\") pod \"iptables-alerter-hxn82\" (UID: \"a63896ac-798f-4830-be8d-76fdfc199e0c\") " pod="openshift-network-operator/iptables-alerter-hxn82" Jul 20 16:45:36.726531 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726467 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-slash\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.726531 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726505 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-etc-kubernetes\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.726646 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726539 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/808cf239-1e7e-484f-89f2-e8e9ad673366-kubelet-config\") pod \"global-pull-secret-syncer-t2zqd\" (UID: \"808cf239-1e7e-484f-89f2-e8e9ad673366\") " pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:36.726646 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726564 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-system-cni-dir\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.726646 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726587 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/81f36e5a-cb50-4407-820e-506a836f18dc-os-release\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.726646 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726589 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/808cf239-1e7e-484f-89f2-e8e9ad673366-kubelet-config\") pod \"global-pull-secret-syncer-t2zqd\" (UID: \"808cf239-1e7e-484f-89f2-e8e9ad673366\") " pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:36.726646 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726611 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-node-log\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.726646 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726640 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-cni-netd\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.726829 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726662 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-sys-fs\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.726829 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726680 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-host-run-k8s-cni-cncf-io\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.726829 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726702 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-socket-dir\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.726829 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726718 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/507c01a0-2197-4761-b183-58de663742c4-hosts-file\") pod \"node-resolver-88g67\" (UID: \"507c01a0-2197-4761-b183-58de663742c4\") " pod="openshift-dns/node-resolver-88g67" Jul 20 16:45:36.726829 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726734 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-etc-sysconfig\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.726829 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726749 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/81f36e5a-cb50-4407-820e-506a836f18dc-cnibin\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.726829 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726765 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-etc-openvswitch\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.727150 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726850 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-registration-dir\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.727150 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726869 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-os-release\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.727150 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726884 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-host-run-multus-certs\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.727150 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726899 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-run\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.727150 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726914 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-var-lib-kubelet\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.727150 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726929 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lht5\" (UniqueName: \"kubernetes.io/projected/9b050c6e-0167-4de3-8e58-3947e6476576-kube-api-access-6lht5\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.727150 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726944 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/48d72ddd-0615-4d29-875a-664eda1f2f88-ovn-node-metrics-cert\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.727150 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726958 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2r9n\" (UniqueName: \"kubernetes.io/projected/48d72ddd-0615-4d29-875a-664eda1f2f88-kube-api-access-l2r9n\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.727150 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.726992 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a6b9b3cd-b414-46f7-9814-6b334f384e70-host\") pod \"node-ca-ps2lf\" (UID: \"a6b9b3cd-b414-46f7-9814-6b334f384e70\") " pod="openshift-image-registry/node-ca-ps2lf" Jul 20 16:45:36.727150 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727015 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-host-var-lib-cni-bin\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.727150 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727031 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-etc-kubernetes\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.727150 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727046 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-etc-modprobe-d\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.727150 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727062 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-sys\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.727150 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727086 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fpsr\" (UniqueName: \"kubernetes.io/projected/81f36e5a-cb50-4407-820e-506a836f18dc-kube-api-access-5fpsr\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.727150 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727108 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/48d72ddd-0615-4d29-875a-664eda1f2f88-env-overrides\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.727150 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727144 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.728037 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727178 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-etc-selinux\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.728037 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727216 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e820f776-0544-43e3-a7f3-de0235215c36-agent-certs\") pod \"konnectivity-agent-vsvts\" (UID: \"e820f776-0544-43e3-a7f3-de0235215c36\") " pod="kube-system/konnectivity-agent-vsvts" Jul 20 16:45:36.728037 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727243 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwpp7\" (UniqueName: \"kubernetes.io/projected/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-kube-api-access-hwpp7\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.728037 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727272 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-265rd\" (UniqueName: \"kubernetes.io/projected/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-kube-api-access-265rd\") pod \"network-metrics-daemon-6xs67\" (UID: \"b18dfe54-9dfb-4d77-b2f5-5fe6837623d5\") " pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:36.728037 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727300 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvtr2\" (UniqueName: \"kubernetes.io/projected/a63896ac-798f-4830-be8d-76fdfc199e0c-kube-api-access-vvtr2\") pod \"iptables-alerter-hxn82\" (UID: \"a63896ac-798f-4830-be8d-76fdfc199e0c\") " pod="openshift-network-operator/iptables-alerter-hxn82" Jul 20 16:45:36.728037 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727325 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/507c01a0-2197-4761-b183-58de663742c4-tmp-dir\") pod \"node-resolver-88g67\" (UID: \"507c01a0-2197-4761-b183-58de663742c4\") " pod="openshift-dns/node-resolver-88g67" Jul 20 16:45:36.728037 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727353 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4tp5\" (UniqueName: \"kubernetes.io/projected/507c01a0-2197-4761-b183-58de663742c4-kube-api-access-k4tp5\") pod \"node-resolver-88g67\" (UID: \"507c01a0-2197-4761-b183-58de663742c4\") " pod="openshift-dns/node-resolver-88g67" Jul 20 16:45:36.728037 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727384 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e820f776-0544-43e3-a7f3-de0235215c36-konnectivity-ca\") pod \"konnectivity-agent-vsvts\" (UID: \"e820f776-0544-43e3-a7f3-de0235215c36\") " pod="kube-system/konnectivity-agent-vsvts" Jul 20 16:45:36.728037 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727415 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/81f36e5a-cb50-4407-820e-506a836f18dc-cni-binary-copy\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.728037 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727442 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/81f36e5a-cb50-4407-820e-506a836f18dc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.728037 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727485 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-systemd-units\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.728037 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727515 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-run-systemd\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.728037 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727544 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/48d72ddd-0615-4d29-875a-664eda1f2f88-ovnkube-config\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.728037 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727548 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 20 16:45:36.728037 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727565 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/48d72ddd-0615-4d29-875a-664eda1f2f88-ovnkube-script-lib\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.728037 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727591 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a63896ac-798f-4830-be8d-76fdfc199e0c-iptables-alerter-script\") pod \"iptables-alerter-hxn82\" (UID: \"a63896ac-798f-4830-be8d-76fdfc199e0c\") " pod="openshift-network-operator/iptables-alerter-hxn82" Jul 20 16:45:36.728797 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727622 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret\") pod \"global-pull-secret-syncer-t2zqd\" (UID: \"808cf239-1e7e-484f-89f2-e8e9ad673366\") " pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:36.728797 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727646 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-cnibin\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.728797 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727671 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b050c6e-0167-4de3-8e58-3947e6476576-tmp\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.728797 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727695 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-cni-bin\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.728797 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727720 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glzwq\" (UniqueName: \"kubernetes.io/projected/a6b9b3cd-b414-46f7-9814-6b334f384e70-kube-api-access-glzwq\") pod \"node-ca-ps2lf\" (UID: \"a6b9b3cd-b414-46f7-9814-6b334f384e70\") " pod="openshift-image-registry/node-ca-ps2lf" Jul 20 16:45:36.728797 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727745 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k952p\" (UniqueName: \"kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p\") pod \"network-check-target-qfqcm\" (UID: \"9ef980cf-2026-4f9a-9e98-b6e02d793100\") " pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:36.728797 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:36.727773 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:36.728797 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727824 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-host-var-lib-kubelet\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.728797 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:36.727874 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret podName:808cf239-1e7e-484f-89f2-e8e9ad673366 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:37.227855604 +0000 UTC m=+2.013854664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret") pod "global-pull-secret-syncer-t2zqd" (UID: "808cf239-1e7e-484f-89f2-e8e9ad673366") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:36.728797 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727894 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-etc-sysctl-d\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.728797 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727913 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-host\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.728797 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727930 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9b050c6e-0167-4de3-8e58-3947e6476576-etc-tuned\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.728797 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727947 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-log-socket\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.728797 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727964 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-multus-conf-dir\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.728797 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727979 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-multus-daemon-config\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.728797 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.727994 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-lib-modules\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.729309 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.728011 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/81f36e5a-cb50-4407-820e-506a836f18dc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.729309 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.728028 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-multus-socket-dir-parent\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.729309 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.728056 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-host-run-netns\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.729309 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.728071 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zbmr\" (UniqueName: \"kubernetes.io/projected/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-kube-api-access-4zbmr\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.729309 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.728095 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-etc-systemd\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.729309 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.728113 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-device-dir\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.729309 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.728128 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-multus-cni-dir\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.729309 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.728144 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-cni-binary-copy\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.729309 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.728168 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-etc-sysctl-conf\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.729309 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.728186 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/81f36e5a-cb50-4407-820e-506a836f18dc-system-cni-dir\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.729309 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.728201 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a6b9b3cd-b414-46f7-9814-6b334f384e70-serviceca\") pod \"node-ca-ps2lf\" (UID: \"a6b9b3cd-b414-46f7-9814-6b334f384e70\") " pod="openshift-image-registry/node-ca-ps2lf" Jul 20 16:45:36.729309 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.728654 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e820f776-0544-43e3-a7f3-de0235215c36-konnectivity-ca\") pod \"konnectivity-agent-vsvts\" (UID: \"e820f776-0544-43e3-a7f3-de0235215c36\") " pod="kube-system/konnectivity-agent-vsvts" Jul 20 16:45:36.731360 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.731337 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e820f776-0544-43e3-a7f3-de0235215c36-agent-certs\") pod \"konnectivity-agent-vsvts\" (UID: \"e820f776-0544-43e3-a7f3-de0235215c36\") " pod="kube-system/konnectivity-agent-vsvts" Jul 20 16:45:36.828473 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828434 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a63896ac-798f-4830-be8d-76fdfc199e0c-iptables-alerter-script\") pod \"iptables-alerter-hxn82\" (UID: \"a63896ac-798f-4830-be8d-76fdfc199e0c\") " pod="openshift-network-operator/iptables-alerter-hxn82" Jul 20 16:45:36.828677 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828500 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-cnibin\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.828677 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828528 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b050c6e-0167-4de3-8e58-3947e6476576-tmp\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.828677 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828549 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-cni-bin\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.828677 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828565 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glzwq\" (UniqueName: \"kubernetes.io/projected/a6b9b3cd-b414-46f7-9814-6b334f384e70-kube-api-access-glzwq\") pod \"node-ca-ps2lf\" (UID: \"a6b9b3cd-b414-46f7-9814-6b334f384e70\") " pod="openshift-image-registry/node-ca-ps2lf" Jul 20 16:45:36.828677 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828584 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k952p\" (UniqueName: \"kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p\") pod \"network-check-target-qfqcm\" (UID: \"9ef980cf-2026-4f9a-9e98-b6e02d793100\") " pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:36.828677 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828606 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-host-var-lib-kubelet\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.828677 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828631 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-etc-sysctl-d\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.828677 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828644 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-cni-bin\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.828677 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828653 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-host\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.828677 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828675 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9b050c6e-0167-4de3-8e58-3947e6476576-etc-tuned\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.829228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828698 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-log-socket\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.829228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828724 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-multus-conf-dir\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.829228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828745 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-multus-daemon-config\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.829228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828747 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-host-var-lib-kubelet\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.829228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828796 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-lib-modules\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.829228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828835 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-host\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.829228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828841 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/81f36e5a-cb50-4407-820e-506a836f18dc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.829228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828865 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-etc-sysctl-d\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.829228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828887 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-multus-socket-dir-parent\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.829228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828912 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-log-socket\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.829228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828916 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-host-run-netns\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.829228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828692 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-cnibin\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.829228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828943 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zbmr\" (UniqueName: \"kubernetes.io/projected/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-kube-api-access-4zbmr\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.829228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828972 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-etc-systemd\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.829228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.828998 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-device-dir\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.829228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829019 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-multus-cni-dir\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.829228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829046 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-cni-binary-copy\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.829228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829077 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-etc-sysctl-conf\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.830144 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829101 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/81f36e5a-cb50-4407-820e-506a836f18dc-system-cni-dir\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.830144 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829135 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a6b9b3cd-b414-46f7-9814-6b334f384e70-serviceca\") pod \"node-ca-ps2lf\" (UID: \"a6b9b3cd-b414-46f7-9814-6b334f384e70\") " pod="openshift-image-registry/node-ca-ps2lf" Jul 20 16:45:36.830144 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829163 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-kubelet\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.830144 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829187 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.830144 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829216 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs\") pod \"network-metrics-daemon-6xs67\" (UID: \"b18dfe54-9dfb-4d77-b2f5-5fe6837623d5\") " pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:36.830144 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829242 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-run-openvswitch\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.830144 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829267 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-run-ovn\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.830144 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829289 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-lib-modules\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.830144 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829346 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-run-ovn-kubernetes\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.830144 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829293 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-run-ovn-kubernetes\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.830144 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829369 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-etc-sysctl-conf\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.830144 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829368 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/81f36e5a-cb50-4407-820e-506a836f18dc-system-cni-dir\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.830144 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829437 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/81f36e5a-cb50-4407-820e-506a836f18dc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.830144 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829440 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-host-run-netns\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.830144 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829466 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-multus-daemon-config\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.830144 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829397 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/81f36e5a-cb50-4407-820e-506a836f18dc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.830144 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829514 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-multus-conf-dir\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.831099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829564 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-device-dir\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.831099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829627 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-multus-socket-dir-parent\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.831099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829186 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a63896ac-798f-4830-be8d-76fdfc199e0c-iptables-alerter-script\") pod \"iptables-alerter-hxn82\" (UID: \"a63896ac-798f-4830-be8d-76fdfc199e0c\") " pod="openshift-network-operator/iptables-alerter-hxn82" Jul 20 16:45:36.831099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829647 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/81f36e5a-cb50-4407-820e-506a836f18dc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.831099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829723 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-etc-systemd\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.831099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829757 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-run-openvswitch\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.831099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829757 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-multus-cni-dir\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.831099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829801 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-run-ovn\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.831099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829817 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-kubelet\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.831099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829816 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.831099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829890 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-run-netns\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.831099 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:36.829893 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:36.831099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829922 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-var-lib-openvswitch\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.831099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829950 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-host-var-lib-cni-multus\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.831099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829952 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-run-netns\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.831099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.829995 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-host-var-lib-cni-multus\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.831099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830017 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-var-lib-openvswitch\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.831975 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:36.830032 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs podName:b18dfe54-9dfb-4d77-b2f5-5fe6837623d5 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:37.330009279 +0000 UTC m=+2.116008352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs") pod "network-metrics-daemon-6xs67" (UID: "b18dfe54-9dfb-4d77-b2f5-5fe6837623d5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:36.831975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830055 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-cni-binary-copy\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.831975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830059 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-hostroot\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.831975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830092 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a63896ac-798f-4830-be8d-76fdfc199e0c-host-slash\") pod \"iptables-alerter-hxn82\" (UID: \"a63896ac-798f-4830-be8d-76fdfc199e0c\") " pod="openshift-network-operator/iptables-alerter-hxn82" Jul 20 16:45:36.831975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830117 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-slash\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.831975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830143 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-etc-kubernetes\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.831975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830161 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-hostroot\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.831975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830176 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-system-cni-dir\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.831975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830023 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a6b9b3cd-b414-46f7-9814-6b334f384e70-serviceca\") pod \"node-ca-ps2lf\" (UID: \"a6b9b3cd-b414-46f7-9814-6b334f384e70\") " pod="openshift-image-registry/node-ca-ps2lf" Jul 20 16:45:36.831975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830203 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/81f36e5a-cb50-4407-820e-506a836f18dc-os-release\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.831975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830233 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-node-log\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.831975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830258 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-cni-netd\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.831975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830262 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-slash\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.831975 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:36.830276 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod976724d148be7ca265b295555429a347.slice/crio-a371c6ed17f02d3aeb38058e15347b779a89f723872b52035a48aa3b1c0d8738 WatchSource:0}: Error finding container a371c6ed17f02d3aeb38058e15347b779a89f723872b52035a48aa3b1c0d8738: Status 404 returned error can't find the container with id a371c6ed17f02d3aeb38058e15347b779a89f723872b52035a48aa3b1c0d8738 Jul 20 16:45:36.831975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830286 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-sys-fs\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.831975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830313 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-host-run-k8s-cni-cncf-io\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.831975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830313 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a63896ac-798f-4830-be8d-76fdfc199e0c-host-slash\") pod \"iptables-alerter-hxn82\" (UID: \"a63896ac-798f-4830-be8d-76fdfc199e0c\") " pod="openshift-network-operator/iptables-alerter-hxn82" Jul 20 16:45:36.832828 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830340 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-socket-dir\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.832828 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830365 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-node-log\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.832828 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830369 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/507c01a0-2197-4761-b183-58de663742c4-hosts-file\") pod \"node-resolver-88g67\" (UID: \"507c01a0-2197-4761-b183-58de663742c4\") " pod="openshift-dns/node-resolver-88g67" Jul 20 16:45:36.832828 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830397 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-etc-sysconfig\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.832828 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830421 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-system-cni-dir\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.832828 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830430 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/81f36e5a-cb50-4407-820e-506a836f18dc-os-release\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.832828 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830423 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/81f36e5a-cb50-4407-820e-506a836f18dc-cnibin\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.832828 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830472 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-host-cni-netd\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.832828 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830483 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-etc-kubernetes\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.832828 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830486 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-etc-openvswitch\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.832828 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830523 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-registration-dir\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.832828 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830551 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-os-release\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.832828 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830555 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-host-run-k8s-cni-cncf-io\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.832828 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830598 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/81f36e5a-cb50-4407-820e-506a836f18dc-cnibin\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.832828 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830601 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-socket-dir\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.832828 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830607 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-sys-fs\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.832828 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:36.830619 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode496a6e153fcce1aa86c117a388fbbcb.slice/crio-48536552dd01be9f0f8aa71854388ff0850c2780e4e2a30186de05a238ff8096 WatchSource:0}: Error finding container 48536552dd01be9f0f8aa71854388ff0850c2780e4e2a30186de05a238ff8096: Status 404 returned error can't find the container with id 48536552dd01be9f0f8aa71854388ff0850c2780e4e2a30186de05a238ff8096 Jul 20 16:45:36.833482 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830624 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-host-run-multus-certs\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.833482 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830654 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-host-run-multus-certs\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.833482 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830660 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-os-release\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.833482 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830655 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-run\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.833482 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830661 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/507c01a0-2197-4761-b183-58de663742c4-hosts-file\") pod \"node-resolver-88g67\" (UID: \"507c01a0-2197-4761-b183-58de663742c4\") " pod="openshift-dns/node-resolver-88g67" Jul 20 16:45:36.833482 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830689 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-run\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.833482 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830695 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-var-lib-kubelet\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.833482 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830705 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-etc-sysconfig\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.833482 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830721 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lht5\" (UniqueName: \"kubernetes.io/projected/9b050c6e-0167-4de3-8e58-3947e6476576-kube-api-access-6lht5\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.833482 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830736 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-var-lib-kubelet\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.833482 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830745 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/48d72ddd-0615-4d29-875a-664eda1f2f88-ovn-node-metrics-cert\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.833482 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830774 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2r9n\" (UniqueName: \"kubernetes.io/projected/48d72ddd-0615-4d29-875a-664eda1f2f88-kube-api-access-l2r9n\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.833482 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830803 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a6b9b3cd-b414-46f7-9814-6b334f384e70-host\") pod \"node-ca-ps2lf\" (UID: \"a6b9b3cd-b414-46f7-9814-6b334f384e70\") " pod="openshift-image-registry/node-ca-ps2lf" Jul 20 16:45:36.833482 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830829 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-host-var-lib-cni-bin\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.833482 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830525 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-etc-openvswitch\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.833482 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830890 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-etc-kubernetes\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.833482 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830917 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-etc-modprobe-d\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.833482 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830943 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-sys\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.834161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830720 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-registration-dir\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.834161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830987 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fpsr\" (UniqueName: \"kubernetes.io/projected/81f36e5a-cb50-4407-820e-506a836f18dc-kube-api-access-5fpsr\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.834161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.830996 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-sys\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.834161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831021 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/48d72ddd-0615-4d29-875a-664eda1f2f88-env-overrides\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.834161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831041 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a6b9b3cd-b414-46f7-9814-6b334f384e70-host\") pod \"node-ca-ps2lf\" (UID: \"a6b9b3cd-b414-46f7-9814-6b334f384e70\") " pod="openshift-image-registry/node-ca-ps2lf" Jul 20 16:45:36.834161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831047 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.834161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831074 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-etc-selinux\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.834161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831083 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-host-var-lib-cni-bin\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.834161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831099 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwpp7\" (UniqueName: \"kubernetes.io/projected/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-kube-api-access-hwpp7\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.834161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831127 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-265rd\" (UniqueName: \"kubernetes.io/projected/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-kube-api-access-265rd\") pod \"network-metrics-daemon-6xs67\" (UID: \"b18dfe54-9dfb-4d77-b2f5-5fe6837623d5\") " pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:36.834161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831120 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-etc-kubernetes\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.834161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831154 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvtr2\" (UniqueName: \"kubernetes.io/projected/a63896ac-798f-4830-be8d-76fdfc199e0c-kube-api-access-vvtr2\") pod \"iptables-alerter-hxn82\" (UID: \"a63896ac-798f-4830-be8d-76fdfc199e0c\") " pod="openshift-network-operator/iptables-alerter-hxn82" Jul 20 16:45:36.834161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831177 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/507c01a0-2197-4761-b183-58de663742c4-tmp-dir\") pod \"node-resolver-88g67\" (UID: \"507c01a0-2197-4761-b183-58de663742c4\") " pod="openshift-dns/node-resolver-88g67" Jul 20 16:45:36.834161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831191 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.834161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831193 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9b050c6e-0167-4de3-8e58-3947e6476576-etc-modprobe-d\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.834161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831206 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4tp5\" (UniqueName: \"kubernetes.io/projected/507c01a0-2197-4761-b183-58de663742c4-kube-api-access-k4tp5\") pod \"node-resolver-88g67\" (UID: \"507c01a0-2197-4761-b183-58de663742c4\") " pod="openshift-dns/node-resolver-88g67" Jul 20 16:45:36.834161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831253 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/81f36e5a-cb50-4407-820e-506a836f18dc-cni-binary-copy\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.834682 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831277 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/81f36e5a-cb50-4407-820e-506a836f18dc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.834682 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831319 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-systemd-units\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.834682 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831343 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-run-systemd\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.834682 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831372 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/48d72ddd-0615-4d29-875a-664eda1f2f88-ovnkube-config\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.834682 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831395 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/48d72ddd-0615-4d29-875a-664eda1f2f88-ovnkube-script-lib\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.834682 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831660 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/48d72ddd-0615-4d29-875a-664eda1f2f88-env-overrides\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.834682 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831718 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b050c6e-0167-4de3-8e58-3947e6476576-tmp\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.834682 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831802 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-systemd-units\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.834682 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831974 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-etc-selinux\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.834682 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.831976 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/48d72ddd-0615-4d29-875a-664eda1f2f88-ovnkube-script-lib\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.834682 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.832019 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/48d72ddd-0615-4d29-875a-664eda1f2f88-run-systemd\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.834682 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.832286 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/81f36e5a-cb50-4407-820e-506a836f18dc-cni-binary-copy\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.834682 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.832358 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/507c01a0-2197-4761-b183-58de663742c4-tmp-dir\") pod \"node-resolver-88g67\" (UID: \"507c01a0-2197-4761-b183-58de663742c4\") " pod="openshift-dns/node-resolver-88g67" Jul 20 16:45:36.834682 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.832420 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9b050c6e-0167-4de3-8e58-3947e6476576-etc-tuned\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.834682 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.832482 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/48d72ddd-0615-4d29-875a-664eda1f2f88-ovnkube-config\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.834682 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.832611 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/81f36e5a-cb50-4407-820e-506a836f18dc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.834682 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.833122 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/48d72ddd-0615-4d29-875a-664eda1f2f88-ovn-node-metrics-cert\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.836340 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:36.836321 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:45:36.836340 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:36.836341 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:45:36.836490 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:36.836352 2566 projected.go:196] Error preparing data for projected volume kube-api-access-k952p for pod openshift-network-diagnostics/network-check-target-qfqcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:36.836490 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:36.836396 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p podName:9ef980cf-2026-4f9a-9e98-b6e02d793100 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:37.336383307 +0000 UTC m=+2.122382388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k952p" (UniqueName: "kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p") pod "network-check-target-qfqcm" (UID: "9ef980cf-2026-4f9a-9e98-b6e02d793100") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:36.837640 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.837605 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 16:45:36.838887 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.838797 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zbmr\" (UniqueName: \"kubernetes.io/projected/d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab-kube-api-access-4zbmr\") pod \"multus-ksj8s\" (UID: \"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab\") " pod="openshift-multus/multus-ksj8s" Jul 20 16:45:36.839167 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.839150 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glzwq\" (UniqueName: \"kubernetes.io/projected/a6b9b3cd-b414-46f7-9814-6b334f384e70-kube-api-access-glzwq\") pod \"node-ca-ps2lf\" (UID: \"a6b9b3cd-b414-46f7-9814-6b334f384e70\") " pod="openshift-image-registry/node-ca-ps2lf" Jul 20 16:45:36.839610 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.839583 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2r9n\" (UniqueName: \"kubernetes.io/projected/48d72ddd-0615-4d29-875a-664eda1f2f88-kube-api-access-l2r9n\") pod \"ovnkube-node-l5n5k\" (UID: \"48d72ddd-0615-4d29-875a-664eda1f2f88\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:36.842038 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.841423 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fpsr\" (UniqueName: \"kubernetes.io/projected/81f36e5a-cb50-4407-820e-506a836f18dc-kube-api-access-5fpsr\") pod \"multus-additional-cni-plugins-27ncp\" (UID: \"81f36e5a-cb50-4407-820e-506a836f18dc\") " pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:36.842038 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.841534 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvtr2\" (UniqueName: \"kubernetes.io/projected/a63896ac-798f-4830-be8d-76fdfc199e0c-kube-api-access-vvtr2\") pod \"iptables-alerter-hxn82\" (UID: \"a63896ac-798f-4830-be8d-76fdfc199e0c\") " pod="openshift-network-operator/iptables-alerter-hxn82" Jul 20 16:45:36.842038 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.841838 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lht5\" (UniqueName: \"kubernetes.io/projected/9b050c6e-0167-4de3-8e58-3947e6476576-kube-api-access-6lht5\") pod \"tuned-xk5jb\" (UID: \"9b050c6e-0167-4de3-8e58-3947e6476576\") " pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:36.842038 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.841995 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4tp5\" (UniqueName: \"kubernetes.io/projected/507c01a0-2197-4761-b183-58de663742c4-kube-api-access-k4tp5\") pod \"node-resolver-88g67\" (UID: \"507c01a0-2197-4761-b183-58de663742c4\") " pod="openshift-dns/node-resolver-88g67" Jul 20 16:45:36.842399 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.842377 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-265rd\" (UniqueName: \"kubernetes.io/projected/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-kube-api-access-265rd\") pod \"network-metrics-daemon-6xs67\" (UID: \"b18dfe54-9dfb-4d77-b2f5-5fe6837623d5\") " pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:36.845393 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.845371 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwpp7\" (UniqueName: \"kubernetes.io/projected/6c649da3-5b5f-49ec-a1c1-1bbde1b66eee-kube-api-access-hwpp7\") pod \"aws-ebs-csi-driver-node-kdvrb\" (UID: \"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.950286 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.950178 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vsvts" Jul 20 16:45:36.960040 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:36.960013 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode820f776_0544_43e3_a7f3_de0235215c36.slice/crio-d5fdd4276561ac019de40d64b94156a77fb28504a7658cef5197ddd3f7d94db4 WatchSource:0}: Error finding container d5fdd4276561ac019de40d64b94156a77fb28504a7658cef5197ddd3f7d94db4: Status 404 returned error can't find the container with id d5fdd4276561ac019de40d64b94156a77fb28504a7658cef5197ddd3f7d94db4 Jul 20 16:45:36.971961 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.971942 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" Jul 20 16:45:36.980225 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:36.980200 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c649da3_5b5f_49ec_a1c1_1bbde1b66eee.slice/crio-fac31733cdab2ea8be030ca84ca8e846e337240423c29ec5799ec0298430712d WatchSource:0}: Error finding container fac31733cdab2ea8be030ca84ca8e846e337240423c29ec5799ec0298430712d: Status 404 returned error can't find the container with id fac31733cdab2ea8be030ca84ca8e846e337240423c29ec5799ec0298430712d Jul 20 16:45:36.988618 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:36.988595 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-88g67" Jul 20 16:45:36.996439 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:36.996416 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod507c01a0_2197_4761_b183_58de663742c4.slice/crio-f3f053f17ca493916c21fc09f9a14cc6d3089be40bbf4d782f6d3be1bda3413e WatchSource:0}: Error finding container f3f053f17ca493916c21fc09f9a14cc6d3089be40bbf4d782f6d3be1bda3413e: Status 404 returned error can't find the container with id f3f053f17ca493916c21fc09f9a14cc6d3089be40bbf4d782f6d3be1bda3413e Jul 20 16:45:37.000728 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.000707 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ps2lf" Jul 20 16:45:37.010349 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:37.010322 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6b9b3cd_b414_46f7_9814_6b334f384e70.slice/crio-5b274731238c84045d844c8ecbb9faa1198bde041d3be67a26c25185161951b3 WatchSource:0}: Error finding container 5b274731238c84045d844c8ecbb9faa1198bde041d3be67a26c25185161951b3: Status 404 returned error can't find the container with id 5b274731238c84045d844c8ecbb9faa1198bde041d3be67a26c25185161951b3 Jul 20 16:45:37.015644 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.015623 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ksj8s" Jul 20 16:45:37.022983 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:37.022961 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4e46a43_6ac5_4cf1_b83a_ccd3b67877ab.slice/crio-53a7beca69abf1ba6c3ab81942edf739662ec3e0096a9c393da741979a70a0dc WatchSource:0}: Error finding container 53a7beca69abf1ba6c3ab81942edf739662ec3e0096a9c393da741979a70a0dc: Status 404 returned error can't find the container with id 53a7beca69abf1ba6c3ab81942edf739662ec3e0096a9c393da741979a70a0dc Jul 20 16:45:37.039246 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.039210 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hxn82" Jul 20 16:45:37.047197 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:37.047174 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda63896ac_798f_4830_be8d_76fdfc199e0c.slice/crio-1cd99f099c24b8b6b0ed31d60a5c53d173e6f8969a4bdaa591722b45a88c3451 WatchSource:0}: Error finding container 1cd99f099c24b8b6b0ed31d60a5c53d173e6f8969a4bdaa591722b45a88c3451: Status 404 returned error can't find the container with id 1cd99f099c24b8b6b0ed31d60a5c53d173e6f8969a4bdaa591722b45a88c3451 Jul 20 16:45:37.051768 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.051748 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" Jul 20 16:45:37.059433 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.059412 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:45:37.059720 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:37.059698 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b050c6e_0167_4de3_8e58_3947e6476576.slice/crio-99b0b41c4f879b7a5351ab1a7c64f3c5c9439543541525f25616cf722b55363b WatchSource:0}: Error finding container 99b0b41c4f879b7a5351ab1a7c64f3c5c9439543541525f25616cf722b55363b: Status 404 returned error can't find the container with id 99b0b41c4f879b7a5351ab1a7c64f3c5c9439543541525f25616cf722b55363b Jul 20 16:45:37.063137 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.063097 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-27ncp" Jul 20 16:45:37.069510 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.069484 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:45:37.071795 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:37.071773 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81f36e5a_cb50_4407_820e_506a836f18dc.slice/crio-0147e50ed8197d0496163ee0ea1173df4ba2224bb82687fd4286b793ef617063 WatchSource:0}: Error finding container 0147e50ed8197d0496163ee0ea1173df4ba2224bb82687fd4286b793ef617063: Status 404 returned error can't find the container with id 0147e50ed8197d0496163ee0ea1173df4ba2224bb82687fd4286b793ef617063 Jul 20 16:45:37.077826 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:45:37.077799 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48d72ddd_0615_4d29_875a_664eda1f2f88.slice/crio-dbd5f12568930950809b2804cead205810704436648f9a3913dbc2f1ca436df0 WatchSource:0}: Error finding container dbd5f12568930950809b2804cead205810704436648f9a3913dbc2f1ca436df0: Status 404 returned error can't find the container with id dbd5f12568930950809b2804cead205810704436648f9a3913dbc2f1ca436df0 Jul 20 16:45:37.234188 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.234086 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret\") pod \"global-pull-secret-syncer-t2zqd\" (UID: \"808cf239-1e7e-484f-89f2-e8e9ad673366\") " pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:37.234345 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:37.234252 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:37.234345 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:37.234318 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret podName:808cf239-1e7e-484f-89f2-e8e9ad673366 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:38.234299977 +0000 UTC m=+3.020299036 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret") pod "global-pull-secret-syncer-t2zqd" (UID: "808cf239-1e7e-484f-89f2-e8e9ad673366") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:37.335566 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.335527 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs\") pod \"network-metrics-daemon-6xs67\" (UID: \"b18dfe54-9dfb-4d77-b2f5-5fe6837623d5\") " pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:37.335739 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:37.335671 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:37.335739 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:37.335724 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs podName:b18dfe54-9dfb-4d77-b2f5-5fe6837623d5 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:38.335709731 +0000 UTC m=+3.121708790 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs") pod "network-metrics-daemon-6xs67" (UID: "b18dfe54-9dfb-4d77-b2f5-5fe6837623d5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:37.436861 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.436792 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k952p\" (UniqueName: \"kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p\") pod \"network-check-target-qfqcm\" (UID: \"9ef980cf-2026-4f9a-9e98-b6e02d793100\") " pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:37.437042 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:37.437010 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:45:37.437042 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:37.437036 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:45:37.437171 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:37.437050 2566 projected.go:196] Error preparing data for projected volume kube-api-access-k952p for pod openshift-network-diagnostics/network-check-target-qfqcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:37.437171 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:37.437107 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p podName:9ef980cf-2026-4f9a-9e98-b6e02d793100 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:38.437091926 +0000 UTC m=+3.223090988 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k952p" (UniqueName: "kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p") pod "network-check-target-qfqcm" (UID: "9ef980cf-2026-4f9a-9e98-b6e02d793100") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:37.672612 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.672538 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 16:40:37 +0000 UTC" deadline="2028-04-19 12:12:08.839024864 +0000 UTC" Jul 20 16:45:37.673370 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.672609 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15331h26m31.166419959s" Jul 20 16:45:37.679984 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.679950 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:45:37.817942 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.817818 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" event={"ID":"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee","Type":"ContainerStarted","Data":"fac31733cdab2ea8be030ca84ca8e846e337240423c29ec5799ec0298430712d"} Jul 20 16:45:37.824542 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.824503 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-18.ec2.internal" event={"ID":"e496a6e153fcce1aa86c117a388fbbcb","Type":"ContainerStarted","Data":"48536552dd01be9f0f8aa71854388ff0850c2780e4e2a30186de05a238ff8096"} Jul 20 16:45:37.829361 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.829306 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" event={"ID":"48d72ddd-0615-4d29-875a-664eda1f2f88","Type":"ContainerStarted","Data":"dbd5f12568930950809b2804cead205810704436648f9a3913dbc2f1ca436df0"} Jul 20 16:45:37.831970 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.831940 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-27ncp" event={"ID":"81f36e5a-cb50-4407-820e-506a836f18dc","Type":"ContainerStarted","Data":"0147e50ed8197d0496163ee0ea1173df4ba2224bb82687fd4286b793ef617063"} Jul 20 16:45:37.834598 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.834566 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ksj8s" event={"ID":"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab","Type":"ContainerStarted","Data":"53a7beca69abf1ba6c3ab81942edf739662ec3e0096a9c393da741979a70a0dc"} Jul 20 16:45:37.836517 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.836486 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-88g67" event={"ID":"507c01a0-2197-4761-b183-58de663742c4","Type":"ContainerStarted","Data":"f3f053f17ca493916c21fc09f9a14cc6d3089be40bbf4d782f6d3be1bda3413e"} Jul 20 16:45:37.838670 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.838642 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vsvts" event={"ID":"e820f776-0544-43e3-a7f3-de0235215c36","Type":"ContainerStarted","Data":"d5fdd4276561ac019de40d64b94156a77fb28504a7658cef5197ddd3f7d94db4"} Jul 20 16:45:37.841075 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.841049 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal" event={"ID":"976724d148be7ca265b295555429a347","Type":"ContainerStarted","Data":"a371c6ed17f02d3aeb38058e15347b779a89f723872b52035a48aa3b1c0d8738"} Jul 20 16:45:37.843443 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.843382 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" event={"ID":"9b050c6e-0167-4de3-8e58-3947e6476576","Type":"ContainerStarted","Data":"99b0b41c4f879b7a5351ab1a7c64f3c5c9439543541525f25616cf722b55363b"} Jul 20 16:45:37.845839 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.845742 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hxn82" event={"ID":"a63896ac-798f-4830-be8d-76fdfc199e0c","Type":"ContainerStarted","Data":"1cd99f099c24b8b6b0ed31d60a5c53d173e6f8969a4bdaa591722b45a88c3451"} Jul 20 16:45:37.853372 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:37.853346 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ps2lf" event={"ID":"a6b9b3cd-b414-46f7-9814-6b334f384e70","Type":"ContainerStarted","Data":"5b274731238c84045d844c8ecbb9faa1198bde041d3be67a26c25185161951b3"} Jul 20 16:45:38.242533 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:38.242486 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret\") pod \"global-pull-secret-syncer-t2zqd\" (UID: \"808cf239-1e7e-484f-89f2-e8e9ad673366\") " pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:38.242758 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:38.242706 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:38.242877 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:38.242777 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret podName:808cf239-1e7e-484f-89f2-e8e9ad673366 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:40.242755068 +0000 UTC m=+5.028754142 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret") pod "global-pull-secret-syncer-t2zqd" (UID: "808cf239-1e7e-484f-89f2-e8e9ad673366") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:38.344675 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:38.343971 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs\") pod \"network-metrics-daemon-6xs67\" (UID: \"b18dfe54-9dfb-4d77-b2f5-5fe6837623d5\") " pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:38.344675 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:38.344192 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:38.344675 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:38.344268 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs podName:b18dfe54-9dfb-4d77-b2f5-5fe6837623d5 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:40.344245838 +0000 UTC m=+5.130244915 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs") pod "network-metrics-daemon-6xs67" (UID: "b18dfe54-9dfb-4d77-b2f5-5fe6837623d5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:38.446043 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:38.445347 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k952p\" (UniqueName: \"kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p\") pod \"network-check-target-qfqcm\" (UID: \"9ef980cf-2026-4f9a-9e98-b6e02d793100\") " pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:38.446043 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:38.445543 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:45:38.446043 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:38.445567 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:45:38.446043 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:38.445581 2566 projected.go:196] Error preparing data for projected volume kube-api-access-k952p for pod openshift-network-diagnostics/network-check-target-qfqcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:38.446043 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:38.445646 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p podName:9ef980cf-2026-4f9a-9e98-b6e02d793100 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:40.445625259 +0000 UTC m=+5.231624332 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k952p" (UniqueName: "kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p") pod "network-check-target-qfqcm" (UID: "9ef980cf-2026-4f9a-9e98-b6e02d793100") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:38.584530 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:38.584316 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:45:38.673381 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:38.673296 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 16:40:37 +0000 UTC" deadline="2028-02-13 12:28:27.782308868 +0000 UTC" Jul 20 16:45:38.673381 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:38.673341 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13747h42m49.108976446s" Jul 20 16:45:38.719290 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:38.719042 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:45:38.810007 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:38.809067 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:38.810007 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:38.809218 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:45:38.810007 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:38.809647 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:38.810007 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:38.809748 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:45:38.810007 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:38.809816 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:38.810007 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:38.809909 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:45:40.261356 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:40.261289 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret\") pod \"global-pull-secret-syncer-t2zqd\" (UID: \"808cf239-1e7e-484f-89f2-e8e9ad673366\") " pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:40.261828 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:40.261507 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:40.261828 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:40.261578 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret podName:808cf239-1e7e-484f-89f2-e8e9ad673366 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:44.261558438 +0000 UTC m=+9.047557514 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret") pod "global-pull-secret-syncer-t2zqd" (UID: "808cf239-1e7e-484f-89f2-e8e9ad673366") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:40.361861 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:40.361777 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs\") pod \"network-metrics-daemon-6xs67\" (UID: \"b18dfe54-9dfb-4d77-b2f5-5fe6837623d5\") " pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:40.362062 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:40.361969 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:40.362473 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:40.362451 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs podName:b18dfe54-9dfb-4d77-b2f5-5fe6837623d5 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:44.362024375 +0000 UTC m=+9.148023437 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs") pod "network-metrics-daemon-6xs67" (UID: "b18dfe54-9dfb-4d77-b2f5-5fe6837623d5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:40.463151 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:40.463107 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k952p\" (UniqueName: \"kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p\") pod \"network-check-target-qfqcm\" (UID: \"9ef980cf-2026-4f9a-9e98-b6e02d793100\") " pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:40.463336 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:40.463317 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:45:40.463383 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:40.463345 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:45:40.463383 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:40.463360 2566 projected.go:196] Error preparing data for projected volume kube-api-access-k952p for pod openshift-network-diagnostics/network-check-target-qfqcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:40.463451 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:40.463421 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p podName:9ef980cf-2026-4f9a-9e98-b6e02d793100 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:44.4634015 +0000 UTC m=+9.249400584 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k952p" (UniqueName: "kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p") pod "network-check-target-qfqcm" (UID: "9ef980cf-2026-4f9a-9e98-b6e02d793100") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:40.809727 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:40.808972 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:40.809727 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:40.809116 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:45:40.809727 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:40.809586 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:40.809727 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:40.809694 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:40.810100 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:40.809809 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:45:40.810100 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:40.809693 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:45:42.809006 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:42.808956 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:42.809549 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:42.809119 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:45:42.809972 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:42.809635 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:42.809972 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:42.809739 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:45:42.809972 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:42.809802 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:42.809972 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:42.809894 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:45:44.296984 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:44.296940 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret\") pod \"global-pull-secret-syncer-t2zqd\" (UID: \"808cf239-1e7e-484f-89f2-e8e9ad673366\") " pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:44.297493 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:44.297155 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:44.297493 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:44.297221 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret podName:808cf239-1e7e-484f-89f2-e8e9ad673366 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:52.297201986 +0000 UTC m=+17.083201048 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret") pod "global-pull-secret-syncer-t2zqd" (UID: "808cf239-1e7e-484f-89f2-e8e9ad673366") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:44.398335 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:44.398293 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs\") pod \"network-metrics-daemon-6xs67\" (UID: \"b18dfe54-9dfb-4d77-b2f5-5fe6837623d5\") " pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:44.398514 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:44.398498 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:44.398572 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:44.398563 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs podName:b18dfe54-9dfb-4d77-b2f5-5fe6837623d5 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:52.398544838 +0000 UTC m=+17.184543901 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs") pod "network-metrics-daemon-6xs67" (UID: "b18dfe54-9dfb-4d77-b2f5-5fe6837623d5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:44.499836 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:44.499223 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k952p\" (UniqueName: \"kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p\") pod \"network-check-target-qfqcm\" (UID: \"9ef980cf-2026-4f9a-9e98-b6e02d793100\") " pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:44.499836 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:44.499390 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:45:44.499836 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:44.499412 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:45:44.499836 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:44.499424 2566 projected.go:196] Error preparing data for projected volume kube-api-access-k952p for pod openshift-network-diagnostics/network-check-target-qfqcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:44.499836 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:44.499489 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p podName:9ef980cf-2026-4f9a-9e98-b6e02d793100 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:52.499469353 +0000 UTC m=+17.285468424 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k952p" (UniqueName: "kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p") pod "network-check-target-qfqcm" (UID: "9ef980cf-2026-4f9a-9e98-b6e02d793100") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:44.808678 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:44.808646 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:44.808886 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:44.808806 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:45:44.808886 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:44.808823 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:44.809020 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:44.808919 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:45:44.809020 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:44.808960 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:44.809140 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:44.809043 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:45:46.808836 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:46.808794 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:46.809281 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:46.808951 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:45:46.809343 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:46.809291 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:46.809391 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:46.809366 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:45:46.809444 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:46.809409 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:46.809492 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:46.809453 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:45:48.809298 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:48.809265 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:48.809710 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:48.809403 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:45:48.809758 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:48.809726 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:48.809849 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:48.809821 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:45:48.809849 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:48.809837 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:48.810044 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:48.809927 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:45:50.809289 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:50.809234 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:50.809748 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:50.809243 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:50.809748 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:50.809374 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:45:50.809748 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:50.809440 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:45:50.809748 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:50.809259 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:50.809748 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:50.809583 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:45:52.362500 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:52.362459 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret\") pod \"global-pull-secret-syncer-t2zqd\" (UID: \"808cf239-1e7e-484f-89f2-e8e9ad673366\") " pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:52.362973 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:52.362622 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:52.362973 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:52.362693 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret podName:808cf239-1e7e-484f-89f2-e8e9ad673366 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:08.362675243 +0000 UTC m=+33.148674323 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret") pod "global-pull-secret-syncer-t2zqd" (UID: "808cf239-1e7e-484f-89f2-e8e9ad673366") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:52.463525 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:52.463485 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs\") pod \"network-metrics-daemon-6xs67\" (UID: \"b18dfe54-9dfb-4d77-b2f5-5fe6837623d5\") " pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:52.463716 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:52.463651 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:52.463797 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:52.463734 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs podName:b18dfe54-9dfb-4d77-b2f5-5fe6837623d5 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:08.463714262 +0000 UTC m=+33.249713318 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs") pod "network-metrics-daemon-6xs67" (UID: "b18dfe54-9dfb-4d77-b2f5-5fe6837623d5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:52.564032 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:52.563992 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k952p\" (UniqueName: \"kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p\") pod \"network-check-target-qfqcm\" (UID: \"9ef980cf-2026-4f9a-9e98-b6e02d793100\") " pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:52.564188 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:52.564161 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:45:52.564188 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:52.564179 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:45:52.564188 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:52.564189 2566 projected.go:196] Error preparing data for projected volume kube-api-access-k952p for pod openshift-network-diagnostics/network-check-target-qfqcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:52.564340 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:52.564243 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p podName:9ef980cf-2026-4f9a-9e98-b6e02d793100 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:08.564225733 +0000 UTC m=+33.350224790 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k952p" (UniqueName: "kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p") pod "network-check-target-qfqcm" (UID: "9ef980cf-2026-4f9a-9e98-b6e02d793100") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:52.808949 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:52.808862 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:52.809111 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:52.808985 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:52.809111 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:52.808998 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:52.809111 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:52.808997 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:45:52.809277 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:52.809130 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:45:52.809277 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:52.809210 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:45:54.808881 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:54.808834 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:54.808881 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:54.808882 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:54.809408 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:54.808858 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:54.809408 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:54.808962 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:45:54.809408 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:54.809027 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:45:54.809408 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:54.809098 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:45:56.809289 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:56.809249 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:56.809717 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:56.809360 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:56.809717 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:56.809373 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:56.809717 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:56.809369 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:45:56.809717 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:56.809476 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:45:56.809717 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:56.809549 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:45:57.902816 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:57.902583 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ksj8s" event={"ID":"d4e46a43-6ac5-4cf1-b83a-ccd3b67877ab","Type":"ContainerStarted","Data":"af402db7e6bc600a1be30e729af7960a0c6d9840b25c49baf683b0fc34ffb696"} Jul 20 16:45:57.904888 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:57.904788 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" event={"ID":"9b050c6e-0167-4de3-8e58-3947e6476576","Type":"ContainerStarted","Data":"0cf00a035587e40dcd6729413373aaa5fe1240f5ab9cbb2e6d37a142dacc7bd0"} Jul 20 16:45:57.906074 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:57.906053 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-18.ec2.internal" event={"ID":"e496a6e153fcce1aa86c117a388fbbcb","Type":"ContainerStarted","Data":"13d1455569dbe5aca16941f95570fad3430db153a772d85be900f6e12867ba1f"} Jul 20 16:45:57.920998 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:57.920951 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ksj8s" podStartSLOduration=1.155564863 podStartE2EDuration="21.920933275s" totalImagesPullingTime="20.765368412s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:57.92035285 +0000 UTC m=+22.706351930" watchObservedRunningTime="2026-07-20 16:45:57.920933275 +0000 UTC m=+22.706932370" Jul 20 16:45:57.934819 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:57.934757 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-18.ec2.internal" podStartSLOduration=20.934742048 podStartE2EDuration="20.934742048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:57.934207725 +0000 UTC m=+22.720206808" watchObservedRunningTime="2026-07-20 16:45:57.934742048 +0000 UTC m=+22.720741126" Jul 20 16:45:58.809786 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.809584 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:45:58.809985 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.809621 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:45:58.809985 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:58.809896 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:45:58.809985 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.809625 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:45:58.809985 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:58.809939 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:45:58.810179 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:45:58.810011 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:45:58.911416 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.911391 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 16:45:58.912134 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.911702 2566 generic.go:356] "Generic (PLEG): container finished" podID="48d72ddd-0615-4d29-875a-664eda1f2f88" containerID="c44a8b25debb9edad9ee8f81a5c08dedae86bb4ce7416516b1be527e2812c15c" exitCode=1 Jul 20 16:45:58.912134 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.911759 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" event={"ID":"48d72ddd-0615-4d29-875a-664eda1f2f88","Type":"ContainerStarted","Data":"ed8cb978f9381b2677877e69bc465ae46dc1f7dba4eff708dfce8598d350d324"} Jul 20 16:45:58.912134 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.911842 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" event={"ID":"48d72ddd-0615-4d29-875a-664eda1f2f88","Type":"ContainerStarted","Data":"8280b39c8088843a29e99a72cd52efc13b006b80bc0777e1db5b7c1502a97b8b"} Jul 20 16:45:58.912134 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.911873 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" event={"ID":"48d72ddd-0615-4d29-875a-664eda1f2f88","Type":"ContainerStarted","Data":"37f46fef128e80b88a60533976be90756ebf8dc31cdb097f0b852a4b6322862d"} Jul 20 16:45:58.912134 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.911905 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" event={"ID":"48d72ddd-0615-4d29-875a-664eda1f2f88","Type":"ContainerStarted","Data":"ca110b3684783e15de00ae5d3ee94bb68ab5560ed85e0a6dfbbeeb5c6871a629"} Jul 20 16:45:58.912134 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.911936 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" event={"ID":"48d72ddd-0615-4d29-875a-664eda1f2f88","Type":"ContainerDied","Data":"c44a8b25debb9edad9ee8f81a5c08dedae86bb4ce7416516b1be527e2812c15c"} Jul 20 16:45:58.912134 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.911966 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" event={"ID":"48d72ddd-0615-4d29-875a-664eda1f2f88","Type":"ContainerStarted","Data":"7333ee7830d246ac8f184078c654cd302fe196027911f7bbe6ca78fd3c439e02"} Jul 20 16:45:58.913270 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.913250 2566 generic.go:356] "Generic (PLEG): container finished" podID="81f36e5a-cb50-4407-820e-506a836f18dc" containerID="90dbe85bde70668b4511d0c57156df445e1dc4687aa35d095329663b84f46754" exitCode=0 Jul 20 16:45:58.913400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.913310 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-27ncp" event={"ID":"81f36e5a-cb50-4407-820e-506a836f18dc","Type":"ContainerDied","Data":"90dbe85bde70668b4511d0c57156df445e1dc4687aa35d095329663b84f46754"} Jul 20 16:45:58.914847 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.914804 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-88g67" event={"ID":"507c01a0-2197-4761-b183-58de663742c4","Type":"ContainerStarted","Data":"2ba4d559a6237d9559a34578b8d89b6ca6b624353a1f24c7ab376ad725fe575e"} Jul 20 16:45:58.916099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.916077 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vsvts" event={"ID":"e820f776-0544-43e3-a7f3-de0235215c36","Type":"ContainerStarted","Data":"16830f48841e4858bb32625cdeb526dba5f9816e12b06d7fd0e95cdaa21f9ad5"} Jul 20 16:45:58.917507 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.917486 2566 generic.go:356] "Generic (PLEG): container finished" podID="976724d148be7ca265b295555429a347" containerID="99f659c0dd07db1f4d9e67cc210647a788d6bcfe4ce53d864cf1918b842869bb" exitCode=0 Jul 20 16:45:58.917583 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.917555 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal" event={"ID":"976724d148be7ca265b295555429a347","Type":"ContainerDied","Data":"99f659c0dd07db1f4d9e67cc210647a788d6bcfe4ce53d864cf1918b842869bb"} Jul 20 16:45:58.919073 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.919040 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ps2lf" event={"ID":"a6b9b3cd-b414-46f7-9814-6b334f384e70","Type":"ContainerStarted","Data":"0a3cec7d07ea6da836f0001b16ac1b1d4e1d444c1e029449dfeeaf06193cfef6"} Jul 20 16:45:58.923072 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.923050 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" event={"ID":"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee","Type":"ContainerStarted","Data":"a7e4a0616edf4a7a4c429d14381f83ec4e0b61050a5924bca57bcdd742660718"} Jul 20 16:45:58.990341 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:58.990246 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-88g67" podStartSLOduration=2.234691651 podStartE2EDuration="22.990230226s" totalImagesPullingTime="20.755538575s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:58.954241461 +0000 UTC m=+23.740240539" watchObservedRunningTime="2026-07-20 16:45:58.990230226 +0000 UTC m=+23.776229366" Jul 20 16:45:59.018873 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:59.018827 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xk5jb" podStartSLOduration=2.325734565 podStartE2EDuration="23.018812242s" totalImagesPullingTime="20.693077677s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:58.989901883 +0000 UTC m=+23.775900963" watchObservedRunningTime="2026-07-20 16:45:59.018812242 +0000 UTC m=+23.804811320" Jul 20 16:45:59.059460 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:59.059410 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vsvts" podStartSLOduration=2.26890785 podStartE2EDuration="23.059394721s" totalImagesPullingTime="20.790486871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:59.01938289 +0000 UTC m=+23.805381969" watchObservedRunningTime="2026-07-20 16:45:59.059394721 +0000 UTC m=+23.845393800" Jul 20 16:45:59.059602 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:59.059568 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ps2lf" podStartSLOduration=10.87385218 podStartE2EDuration="23.059564566s" totalImagesPullingTime="12.185712386s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:59.05922863 +0000 UTC m=+23.845227708" watchObservedRunningTime="2026-07-20 16:45:59.059564566 +0000 UTC m=+23.845563678" Jul 20 16:45:59.926656 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:59.926618 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hxn82" event={"ID":"a63896ac-798f-4830-be8d-76fdfc199e0c","Type":"ContainerStarted","Data":"1a5f0827110276e649a3706a841fc7cd900ce31eb432ae6cabeef486a3dcdf5b"} Jul 20 16:45:59.957826 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:45:59.957558 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hxn82" podStartSLOduration=3.281530445 podStartE2EDuration="23.957535384s" totalImagesPullingTime="20.676004939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:59.957531266 +0000 UTC m=+24.743530344" watchObservedRunningTime="2026-07-20 16:45:59.957535384 +0000 UTC m=+24.743534464" Jul 20 16:46:00.070554 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:00.070521 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 20 16:46:00.485979 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:00.485946 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-88g67_507c01a0-2197-4761-b183-58de663742c4/dns-node-resolver/0.log" Jul 20 16:46:00.686098 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:00.686005 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T16:46:00.070549177Z","UUID":"f46a5ba8-c0b0-405c-8539-ea59bbecb496","Handler":null,"Name":"","Endpoint":""} Jul 20 16:46:00.687590 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:00.687560 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 20 16:46:00.687710 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:00.687607 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 20 16:46:00.809453 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:00.809372 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:00.809584 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:00.809494 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:46:00.809584 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:00.809505 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:00.809584 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:00.809533 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:00.809708 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:00.809599 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:46:00.809708 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:00.809648 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:46:00.855116 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:00.855094 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vsvts" Jul 20 16:46:00.855764 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:00.855741 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vsvts" Jul 20 16:46:00.931901 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:00.931875 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 16:46:00.932358 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:00.932326 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" event={"ID":"48d72ddd-0615-4d29-875a-664eda1f2f88","Type":"ContainerStarted","Data":"f9378d19a21e059fd68fe4bf24959e85bec23e0676a6fd1af09e73d1d5c8aa6d"} Jul 20 16:46:00.934049 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:00.934026 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal" event={"ID":"976724d148be7ca265b295555429a347","Type":"ContainerStarted","Data":"51bca59729027a66d749204e27abc2309df3adc06dd641f9d5688c9500598644"} Jul 20 16:46:00.936865 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:00.936840 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" event={"ID":"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee","Type":"ContainerStarted","Data":"1a9d40717c8a6d89287cf670d2ad50f6f81085bb6acc012b26ed4da62e8dd94e"} Jul 20 16:46:00.936964 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:00.936899 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vsvts" Jul 20 16:46:00.937381 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:00.937361 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vsvts" Jul 20 16:46:00.949038 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:00.948989 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-18.ec2.internal" podStartSLOduration=23.948968097 podStartE2EDuration="23.948968097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:00.948906827 +0000 UTC m=+25.734905920" watchObservedRunningTime="2026-07-20 16:46:00.948968097 +0000 UTC m=+25.734967177" Jul 20 16:46:01.462986 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:01.462959 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ps2lf_a6b9b3cd-b414-46f7-9814-6b334f384e70/node-ca/0.log" Jul 20 16:46:01.940835 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:01.940803 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" event={"ID":"6c649da3-5b5f-49ec-a1c1-1bbde1b66eee","Type":"ContainerStarted","Data":"fbdf4a53bb672a6e704055c519a89bd9f5e89ecd1fd96cf2dc7dc1f8298c036a"} Jul 20 16:46:01.958319 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:01.958266 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdvrb" podStartSLOduration=2.591181726 podStartE2EDuration="25.958245787s" totalImagesPullingTime="23.367064061s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:01.958071125 +0000 UTC m=+26.744070204" watchObservedRunningTime="2026-07-20 16:46:01.958245787 +0000 UTC m=+26.744244867" Jul 20 16:46:02.808786 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:02.808726 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:02.808996 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:02.808808 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:02.808996 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:02.808873 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:02.808996 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:02.808889 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:46:02.808996 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:02.808974 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:46:02.809196 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:02.809068 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:46:04.809006 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:04.808837 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:04.809592 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:04.808839 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:04.809592 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:04.809072 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:46:04.809592 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:04.809152 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:46:04.809592 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:04.808839 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:04.809592 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:04.809219 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:46:04.950521 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:04.950495 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 16:46:04.950868 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:04.950846 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" event={"ID":"48d72ddd-0615-4d29-875a-664eda1f2f88","Type":"ContainerStarted","Data":"6e98d6a14d5b0a55e218cd48fb97fe92ca24da69b41e6ccbc0d29f2d352ca684"} Jul 20 16:46:04.951136 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:04.951108 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:46:04.951287 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:04.951183 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:46:04.951287 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:04.951229 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:46:04.951399 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:04.951301 2566 scope.go:117] "RemoveContainer" containerID="c44a8b25debb9edad9ee8f81a5c08dedae86bb4ce7416516b1be527e2812c15c" Jul 20 16:46:04.953069 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:04.953043 2566 generic.go:356] "Generic (PLEG): container finished" podID="81f36e5a-cb50-4407-820e-506a836f18dc" containerID="657c9c77c1d3c455eb43b646311d444e62dbd312a24b22c014c9fcd918f86c53" exitCode=0 Jul 20 16:46:04.953161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:04.953105 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-27ncp" event={"ID":"81f36e5a-cb50-4407-820e-506a836f18dc","Type":"ContainerDied","Data":"657c9c77c1d3c455eb43b646311d444e62dbd312a24b22c014c9fcd918f86c53"} Jul 20 16:46:04.968321 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:04.968301 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:46:04.968412 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:04.968398 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:46:05.959146 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:05.959115 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 16:46:05.959716 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:05.959441 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" event={"ID":"48d72ddd-0615-4d29-875a-664eda1f2f88","Type":"ContainerStarted","Data":"8ba145c2b80d43f16feb5ba4f3e9e8226224cb1f6b6db462a8123429445bd2b8"} Jul 20 16:46:06.010132 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:06.010067 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" podStartSLOduration=9.061761722 podStartE2EDuration="30.010049129s" totalImagesPullingTime="20.948287407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:06.009541272 +0000 UTC m=+30.795540352" watchObservedRunningTime="2026-07-20 16:46:06.010049129 +0000 UTC m=+30.796048207" Jul 20 16:46:06.809000 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:06.808961 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:06.809204 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:06.809105 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:06.809204 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:06.809112 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:46:06.809204 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:06.809115 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:06.809361 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:06.809215 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:46:06.809361 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:06.809310 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:46:06.964167 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:06.964135 2566 generic.go:356] "Generic (PLEG): container finished" podID="81f36e5a-cb50-4407-820e-506a836f18dc" containerID="538552b6e106d0181e21a3147822532787b7d383db77fc11c728cbe4bec01b34" exitCode=0 Jul 20 16:46:06.964613 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:06.964219 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-27ncp" event={"ID":"81f36e5a-cb50-4407-820e-506a836f18dc","Type":"ContainerDied","Data":"538552b6e106d0181e21a3147822532787b7d383db77fc11c728cbe4bec01b34"} Jul 20 16:46:08.377697 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:08.377664 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret\") pod \"global-pull-secret-syncer-t2zqd\" (UID: \"808cf239-1e7e-484f-89f2-e8e9ad673366\") " pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:08.378090 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:08.377845 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:46:08.378090 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:08.377898 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret podName:808cf239-1e7e-484f-89f2-e8e9ad673366 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:40.37788357 +0000 UTC m=+65.163882627 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret") pod "global-pull-secret-syncer-t2zqd" (UID: "808cf239-1e7e-484f-89f2-e8e9ad673366") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:46:08.478716 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:08.478684 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs\") pod \"network-metrics-daemon-6xs67\" (UID: \"b18dfe54-9dfb-4d77-b2f5-5fe6837623d5\") " pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:08.478893 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:08.478810 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:46:08.478893 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:08.478863 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs podName:b18dfe54-9dfb-4d77-b2f5-5fe6837623d5 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:40.47884867 +0000 UTC m=+65.264847728 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs") pod "network-metrics-daemon-6xs67" (UID: "b18dfe54-9dfb-4d77-b2f5-5fe6837623d5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:46:08.579592 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:08.579563 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k952p\" (UniqueName: \"kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p\") pod \"network-check-target-qfqcm\" (UID: \"9ef980cf-2026-4f9a-9e98-b6e02d793100\") " pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:08.579747 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:08.579686 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:46:08.579747 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:08.579699 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:46:08.579747 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:08.579707 2566 projected.go:196] Error preparing data for projected volume kube-api-access-k952p for pod openshift-network-diagnostics/network-check-target-qfqcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:46:08.579868 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:08.579752 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p podName:9ef980cf-2026-4f9a-9e98-b6e02d793100 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:40.579739659 +0000 UTC m=+65.365738716 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k952p" (UniqueName: "kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p") pod "network-check-target-qfqcm" (UID: "9ef980cf-2026-4f9a-9e98-b6e02d793100") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:46:08.809099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:08.809007 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:08.809099 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:08.809059 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:08.809274 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:08.809018 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:08.809274 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:08.809163 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:46:08.809274 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:08.809224 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:46:08.809361 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:08.809315 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:46:08.971506 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:08.971473 2566 generic.go:356] "Generic (PLEG): container finished" podID="81f36e5a-cb50-4407-820e-506a836f18dc" containerID="e8a276c135977a614ea922cdc6008f9767195d48d4680e70873349a6af972701" exitCode=0 Jul 20 16:46:08.971691 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:08.971518 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-27ncp" event={"ID":"81f36e5a-cb50-4407-820e-506a836f18dc","Type":"ContainerDied","Data":"e8a276c135977a614ea922cdc6008f9767195d48d4680e70873349a6af972701"} Jul 20 16:46:10.809275 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:10.809240 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:10.809275 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:10.809265 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:10.809792 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:10.809279 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:10.809792 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:10.809365 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:46:10.809792 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:10.809425 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:46:10.809792 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:10.809501 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:46:12.808863 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:12.808820 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:12.808863 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:12.808856 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:12.809380 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:12.808853 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:12.809380 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:12.808964 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:46:12.809380 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:12.809028 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:46:12.809380 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:12.809109 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:46:13.986135 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:13.985878 2566 generic.go:356] "Generic (PLEG): container finished" podID="81f36e5a-cb50-4407-820e-506a836f18dc" containerID="3c60f23887354f23b1c5e27069b980fb05a1c824b2cd759412663bbdffeace27" exitCode=0 Jul 20 16:46:13.986135 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:13.985970 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-27ncp" event={"ID":"81f36e5a-cb50-4407-820e-506a836f18dc","Type":"ContainerDied","Data":"3c60f23887354f23b1c5e27069b980fb05a1c824b2cd759412663bbdffeace27"} Jul 20 16:46:14.809197 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:14.809163 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:14.809197 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:14.809195 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:14.809392 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:14.809163 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:14.809392 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:14.809281 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:46:14.809392 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:14.809374 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:46:14.809511 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:14.809446 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:46:14.991052 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:14.991022 2566 generic.go:356] "Generic (PLEG): container finished" podID="81f36e5a-cb50-4407-820e-506a836f18dc" containerID="55d1471afd57415121d2c7b86ea92c363ace1bce4ac766bb7af77c169737e2d5" exitCode=0 Jul 20 16:46:14.991375 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:14.991071 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-27ncp" event={"ID":"81f36e5a-cb50-4407-820e-506a836f18dc","Type":"ContainerDied","Data":"55d1471afd57415121d2c7b86ea92c363ace1bce4ac766bb7af77c169737e2d5"} Jul 20 16:46:15.996854 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:15.996817 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-27ncp" event={"ID":"81f36e5a-cb50-4407-820e-506a836f18dc","Type":"ContainerStarted","Data":"a11eec0155c9677cd9587e902a3a43826944b05e746e264bb01d28e05165f4a2"} Jul 20 16:46:16.018400 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:16.018358 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-27ncp" podStartSLOduration=7.3525833800000004 podStartE2EDuration="40.018343734s" totalImagesPullingTime="32.665760354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:16.017983108 +0000 UTC m=+40.803982187" watchObservedRunningTime="2026-07-20 16:46:16.018343734 +0000 UTC m=+40.804342813" Jul 20 16:46:16.809214 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:16.809177 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:16.809381 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:16.809284 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:16.809381 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:16.809288 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:46:16.809381 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:16.809287 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:16.809488 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:16.809380 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:46:16.809488 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:16.809430 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:46:18.809204 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:18.809168 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:18.809204 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:18.809195 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:18.809604 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:18.809229 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:18.809604 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:18.809324 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:46:18.809604 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:18.809451 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:46:18.809604 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:18.809493 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:46:20.809507 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:20.809470 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:20.809978 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:20.809471 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:20.809978 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:20.809586 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:46:20.809978 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:20.809471 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:20.809978 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:20.809692 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:46:20.809978 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:20.809735 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:46:22.809376 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:22.809345 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:22.809949 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:22.809353 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:22.809949 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:22.809456 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:46:22.809949 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:22.809351 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:22.809949 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:22.809565 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:46:22.809949 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:22.809672 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:46:23.295970 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:23.295939 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qfqcm"] Jul 20 16:46:23.296152 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:23.296046 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:23.296226 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:23.296155 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:46:23.299486 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:23.299458 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6xs67"] Jul 20 16:46:23.299623 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:23.299561 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:23.299686 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:23.299652 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:46:23.304177 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:23.301099 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t2zqd"] Jul 20 16:46:23.304177 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:23.301218 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:23.304177 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:23.301347 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:46:24.809502 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:24.809468 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:24.809912 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:24.809574 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:24.809912 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:24.809580 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:24.809912 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:24.809667 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qfqcm" podUID="9ef980cf-2026-4f9a-9e98-b6e02d793100" Jul 20 16:46:24.809912 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:24.809575 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2zqd" podUID="808cf239-1e7e-484f-89f2-e8e9ad673366" Jul 20 16:46:24.809912 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:46:24.809746 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6xs67" podUID="b18dfe54-9dfb-4d77-b2f5-5fe6837623d5" Jul 20 16:46:26.544492 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.544313 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-18.ec2.internal" event="NodeReady" Jul 20 16:46:26.544884 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.544582 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 20 16:46:26.588508 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.588475 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dwlfm"] Jul 20 16:46:26.609328 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.609292 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2xvv9"] Jul 20 16:46:26.609476 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.609424 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dwlfm" Jul 20 16:46:26.612145 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.612115 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 20 16:46:26.612490 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.612213 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vsgmd\"" Jul 20 16:46:26.612490 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.612236 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 20 16:46:26.629076 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.629046 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dwlfm"] Jul 20 16:46:26.629199 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.629084 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2xvv9"] Jul 20 16:46:26.629199 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.629153 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2xvv9" Jul 20 16:46:26.631521 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.631496 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 20 16:46:26.632006 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.631977 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 20 16:46:26.632137 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.632119 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-47jm7\"" Jul 20 16:46:26.632189 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.632155 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 20 16:46:26.688540 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.688507 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tdgwh"] Jul 20 16:46:26.710946 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.710916 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tdgwh"] Jul 20 16:46:26.711090 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.711032 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tdgwh" Jul 20 16:46:26.713155 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.713128 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 20 16:46:26.713349 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.713330 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 20 16:46:26.713460 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.713348 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r489m\" (UniqueName: \"kubernetes.io/projected/1014a924-f279-4b0f-b362-4f39c0c03be0-kube-api-access-r489m\") pod \"dns-default-dwlfm\" (UID: \"1014a924-f279-4b0f-b362-4f39c0c03be0\") " pod="openshift-dns/dns-default-dwlfm" Jul 20 16:46:26.713460 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.713370 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 20 16:46:26.713460 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.713406 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1014a924-f279-4b0f-b362-4f39c0c03be0-config-volume\") pod \"dns-default-dwlfm\" (UID: \"1014a924-f279-4b0f-b362-4f39c0c03be0\") " pod="openshift-dns/dns-default-dwlfm" Jul 20 16:46:26.713460 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.713454 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1014a924-f279-4b0f-b362-4f39c0c03be0-tmp-dir\") pod \"dns-default-dwlfm\" (UID: \"1014a924-f279-4b0f-b362-4f39c0c03be0\") " pod="openshift-dns/dns-default-dwlfm" Jul 20 16:46:26.713677 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.713477 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 20 16:46:26.713677 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.713513 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tcv9s\"" Jul 20 16:46:26.713677 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.713537 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1014a924-f279-4b0f-b362-4f39c0c03be0-metrics-tls\") pod \"dns-default-dwlfm\" (UID: \"1014a924-f279-4b0f-b362-4f39c0c03be0\") " pod="openshift-dns/dns-default-dwlfm" Jul 20 16:46:26.809502 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.809464 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:26.809502 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.809482 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:26.809502 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.809482 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:26.811875 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.811853 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 16:46:26.811875 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.811881 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 16:46:26.812077 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.811986 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9vk4l\"" Jul 20 16:46:26.812077 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.812034 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 16:46:26.812077 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.812034 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 16:46:26.812366 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.812351 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2sqpl\"" Jul 20 16:46:26.813836 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.813813 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jwwc\" (UniqueName: \"kubernetes.io/projected/98ea33d6-9c67-4e13-842a-caffa292568c-kube-api-access-6jwwc\") pod \"insights-runtime-extractor-tdgwh\" (UID: \"98ea33d6-9c67-4e13-842a-caffa292568c\") " pod="openshift-insights/insights-runtime-extractor-tdgwh" Jul 20 16:46:26.813958 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.813863 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1014a924-f279-4b0f-b362-4f39c0c03be0-metrics-tls\") pod \"dns-default-dwlfm\" (UID: \"1014a924-f279-4b0f-b362-4f39c0c03be0\") " pod="openshift-dns/dns-default-dwlfm" Jul 20 16:46:26.813958 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.813915 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/98ea33d6-9c67-4e13-842a-caffa292568c-crio-socket\") pod \"insights-runtime-extractor-tdgwh\" (UID: \"98ea33d6-9c67-4e13-842a-caffa292568c\") " pod="openshift-insights/insights-runtime-extractor-tdgwh" Jul 20 16:46:26.813958 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.813939 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98ea33d6-9c67-4e13-842a-caffa292568c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tdgwh\" (UID: \"98ea33d6-9c67-4e13-842a-caffa292568c\") " pod="openshift-insights/insights-runtime-extractor-tdgwh" Jul 20 16:46:26.814116 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.813982 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r489m\" (UniqueName: \"kubernetes.io/projected/1014a924-f279-4b0f-b362-4f39c0c03be0-kube-api-access-r489m\") pod \"dns-default-dwlfm\" (UID: \"1014a924-f279-4b0f-b362-4f39c0c03be0\") " pod="openshift-dns/dns-default-dwlfm" Jul 20 16:46:26.814116 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.814000 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98ea33d6-9c67-4e13-842a-caffa292568c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tdgwh\" (UID: \"98ea33d6-9c67-4e13-842a-caffa292568c\") " pod="openshift-insights/insights-runtime-extractor-tdgwh" Jul 20 16:46:26.814116 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.814020 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjppk\" (UniqueName: \"kubernetes.io/projected/5b354580-bc22-4283-bbff-5e9e87fc026a-kube-api-access-hjppk\") pod \"ingress-canary-2xvv9\" (UID: \"5b354580-bc22-4283-bbff-5e9e87fc026a\") " pod="openshift-ingress-canary/ingress-canary-2xvv9" Jul 20 16:46:26.814116 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.814056 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1014a924-f279-4b0f-b362-4f39c0c03be0-config-volume\") pod \"dns-default-dwlfm\" (UID: \"1014a924-f279-4b0f-b362-4f39c0c03be0\") " pod="openshift-dns/dns-default-dwlfm" Jul 20 16:46:26.814116 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.814074 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b354580-bc22-4283-bbff-5e9e87fc026a-cert\") pod \"ingress-canary-2xvv9\" (UID: \"5b354580-bc22-4283-bbff-5e9e87fc026a\") " pod="openshift-ingress-canary/ingress-canary-2xvv9" Jul 20 16:46:26.814116 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.814097 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1014a924-f279-4b0f-b362-4f39c0c03be0-tmp-dir\") pod \"dns-default-dwlfm\" (UID: \"1014a924-f279-4b0f-b362-4f39c0c03be0\") " pod="openshift-dns/dns-default-dwlfm" Jul 20 16:46:26.814350 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.814127 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/98ea33d6-9c67-4e13-842a-caffa292568c-data-volume\") pod \"insights-runtime-extractor-tdgwh\" (UID: \"98ea33d6-9c67-4e13-842a-caffa292568c\") " pod="openshift-insights/insights-runtime-extractor-tdgwh" Jul 20 16:46:26.814477 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.814458 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1014a924-f279-4b0f-b362-4f39c0c03be0-tmp-dir\") pod \"dns-default-dwlfm\" (UID: \"1014a924-f279-4b0f-b362-4f39c0c03be0\") " pod="openshift-dns/dns-default-dwlfm" Jul 20 16:46:26.814679 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.814660 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1014a924-f279-4b0f-b362-4f39c0c03be0-config-volume\") pod \"dns-default-dwlfm\" (UID: \"1014a924-f279-4b0f-b362-4f39c0c03be0\") " pod="openshift-dns/dns-default-dwlfm" Jul 20 16:46:26.818896 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.818870 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1014a924-f279-4b0f-b362-4f39c0c03be0-metrics-tls\") pod \"dns-default-dwlfm\" (UID: \"1014a924-f279-4b0f-b362-4f39c0c03be0\") " pod="openshift-dns/dns-default-dwlfm" Jul 20 16:46:26.821395 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.821376 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r489m\" (UniqueName: \"kubernetes.io/projected/1014a924-f279-4b0f-b362-4f39c0c03be0-kube-api-access-r489m\") pod \"dns-default-dwlfm\" (UID: \"1014a924-f279-4b0f-b362-4f39c0c03be0\") " pod="openshift-dns/dns-default-dwlfm" Jul 20 16:46:26.915185 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.915089 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjppk\" (UniqueName: \"kubernetes.io/projected/5b354580-bc22-4283-bbff-5e9e87fc026a-kube-api-access-hjppk\") pod \"ingress-canary-2xvv9\" (UID: \"5b354580-bc22-4283-bbff-5e9e87fc026a\") " pod="openshift-ingress-canary/ingress-canary-2xvv9" Jul 20 16:46:26.915185 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.915125 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b354580-bc22-4283-bbff-5e9e87fc026a-cert\") pod \"ingress-canary-2xvv9\" (UID: \"5b354580-bc22-4283-bbff-5e9e87fc026a\") " pod="openshift-ingress-canary/ingress-canary-2xvv9" Jul 20 16:46:26.915185 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.915159 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/98ea33d6-9c67-4e13-842a-caffa292568c-data-volume\") pod \"insights-runtime-extractor-tdgwh\" (UID: \"98ea33d6-9c67-4e13-842a-caffa292568c\") " pod="openshift-insights/insights-runtime-extractor-tdgwh" Jul 20 16:46:26.915185 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.915180 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jwwc\" (UniqueName: \"kubernetes.io/projected/98ea33d6-9c67-4e13-842a-caffa292568c-kube-api-access-6jwwc\") pod \"insights-runtime-extractor-tdgwh\" (UID: \"98ea33d6-9c67-4e13-842a-caffa292568c\") " pod="openshift-insights/insights-runtime-extractor-tdgwh" Jul 20 16:46:26.915556 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.915227 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/98ea33d6-9c67-4e13-842a-caffa292568c-crio-socket\") pod \"insights-runtime-extractor-tdgwh\" (UID: \"98ea33d6-9c67-4e13-842a-caffa292568c\") " pod="openshift-insights/insights-runtime-extractor-tdgwh" Jul 20 16:46:26.915556 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.915250 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98ea33d6-9c67-4e13-842a-caffa292568c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tdgwh\" (UID: \"98ea33d6-9c67-4e13-842a-caffa292568c\") " pod="openshift-insights/insights-runtime-extractor-tdgwh" Jul 20 16:46:26.915556 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.915287 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98ea33d6-9c67-4e13-842a-caffa292568c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tdgwh\" (UID: \"98ea33d6-9c67-4e13-842a-caffa292568c\") " pod="openshift-insights/insights-runtime-extractor-tdgwh" Jul 20 16:46:26.915556 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.915472 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/98ea33d6-9c67-4e13-842a-caffa292568c-crio-socket\") pod \"insights-runtime-extractor-tdgwh\" (UID: \"98ea33d6-9c67-4e13-842a-caffa292568c\") " pod="openshift-insights/insights-runtime-extractor-tdgwh" Jul 20 16:46:26.915556 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.915498 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/98ea33d6-9c67-4e13-842a-caffa292568c-data-volume\") pod \"insights-runtime-extractor-tdgwh\" (UID: \"98ea33d6-9c67-4e13-842a-caffa292568c\") " pod="openshift-insights/insights-runtime-extractor-tdgwh" Jul 20 16:46:26.915841 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.915822 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98ea33d6-9c67-4e13-842a-caffa292568c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tdgwh\" (UID: \"98ea33d6-9c67-4e13-842a-caffa292568c\") " pod="openshift-insights/insights-runtime-extractor-tdgwh" Jul 20 16:46:26.917796 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.917742 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98ea33d6-9c67-4e13-842a-caffa292568c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tdgwh\" (UID: \"98ea33d6-9c67-4e13-842a-caffa292568c\") " pod="openshift-insights/insights-runtime-extractor-tdgwh" Jul 20 16:46:26.917949 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.917928 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b354580-bc22-4283-bbff-5e9e87fc026a-cert\") pod \"ingress-canary-2xvv9\" (UID: \"5b354580-bc22-4283-bbff-5e9e87fc026a\") " pod="openshift-ingress-canary/ingress-canary-2xvv9" Jul 20 16:46:26.920734 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.920718 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dwlfm" Jul 20 16:46:26.923593 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.923564 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jwwc\" (UniqueName: \"kubernetes.io/projected/98ea33d6-9c67-4e13-842a-caffa292568c-kube-api-access-6jwwc\") pod \"insights-runtime-extractor-tdgwh\" (UID: \"98ea33d6-9c67-4e13-842a-caffa292568c\") " pod="openshift-insights/insights-runtime-extractor-tdgwh" Jul 20 16:46:26.923684 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.923656 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjppk\" (UniqueName: \"kubernetes.io/projected/5b354580-bc22-4283-bbff-5e9e87fc026a-kube-api-access-hjppk\") pod \"ingress-canary-2xvv9\" (UID: \"5b354580-bc22-4283-bbff-5e9e87fc026a\") " pod="openshift-ingress-canary/ingress-canary-2xvv9" Jul 20 16:46:26.937455 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:26.937414 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2xvv9" Jul 20 16:46:27.019917 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:27.019877 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tdgwh" Jul 20 16:46:27.146432 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:27.146393 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2xvv9"] Jul 20 16:46:27.149327 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:27.149303 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dwlfm"] Jul 20 16:46:27.151760 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:46:27.151727 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b354580_bc22_4283_bbff_5e9e87fc026a.slice/crio-79ddf12f918e7f393eca798b2b88e1c56be73b1d4df29e6d9b8cfcd7d62617f0 WatchSource:0}: Error finding container 79ddf12f918e7f393eca798b2b88e1c56be73b1d4df29e6d9b8cfcd7d62617f0: Status 404 returned error can't find the container with id 79ddf12f918e7f393eca798b2b88e1c56be73b1d4df29e6d9b8cfcd7d62617f0 Jul 20 16:46:27.152638 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:46:27.152610 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1014a924_f279_4b0f_b362_4f39c0c03be0.slice/crio-b4cde419f76341c2046d5d97225ff98dbf94c5befbd77e95014fa9120a965329 WatchSource:0}: Error finding container b4cde419f76341c2046d5d97225ff98dbf94c5befbd77e95014fa9120a965329: Status 404 returned error can't find the container with id b4cde419f76341c2046d5d97225ff98dbf94c5befbd77e95014fa9120a965329 Jul 20 16:46:27.192239 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:27.192154 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tdgwh"] Jul 20 16:46:27.198310 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:46:27.198286 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98ea33d6_9c67_4e13_842a_caffa292568c.slice/crio-20ff97d8a9e555e0ab3a479f9fd333b9580759ccadaa821ec8a4e97f2d82d974 WatchSource:0}: Error finding container 20ff97d8a9e555e0ab3a479f9fd333b9580759ccadaa821ec8a4e97f2d82d974: Status 404 returned error can't find the container with id 20ff97d8a9e555e0ab3a479f9fd333b9580759ccadaa821ec8a4e97f2d82d974 Jul 20 16:46:28.027262 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:28.027106 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tdgwh" event={"ID":"98ea33d6-9c67-4e13-842a-caffa292568c","Type":"ContainerStarted","Data":"20ff97d8a9e555e0ab3a479f9fd333b9580759ccadaa821ec8a4e97f2d82d974"} Jul 20 16:46:28.030704 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:28.030624 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2xvv9" event={"ID":"5b354580-bc22-4283-bbff-5e9e87fc026a","Type":"ContainerStarted","Data":"79ddf12f918e7f393eca798b2b88e1c56be73b1d4df29e6d9b8cfcd7d62617f0"} Jul 20 16:46:28.032638 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:28.032602 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dwlfm" event={"ID":"1014a924-f279-4b0f-b362-4f39c0c03be0","Type":"ContainerStarted","Data":"b4cde419f76341c2046d5d97225ff98dbf94c5befbd77e95014fa9120a965329"} Jul 20 16:46:29.791878 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:29.791609 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrdtm"] Jul 20 16:46:29.801409 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:29.801381 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrdtm" Jul 20 16:46:29.801560 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:29.801520 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrdtm"] Jul 20 16:46:29.805040 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:29.804798 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 20 16:46:29.805040 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:29.804836 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-mhd95\"" Jul 20 16:46:29.939526 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:29.939487 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/946f07eb-462e-48d7-a469-692c829dfabf-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-zrdtm\" (UID: \"946f07eb-462e-48d7-a469-692c829dfabf\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrdtm" Jul 20 16:46:30.040669 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:30.040615 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/946f07eb-462e-48d7-a469-692c829dfabf-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-zrdtm\" (UID: \"946f07eb-462e-48d7-a469-692c829dfabf\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrdtm" Jul 20 16:46:30.045422 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:30.045312 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/946f07eb-462e-48d7-a469-692c829dfabf-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-zrdtm\" (UID: \"946f07eb-462e-48d7-a469-692c829dfabf\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrdtm" Jul 20 16:46:30.115376 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:30.115333 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrdtm" Jul 20 16:46:33.368287 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:33.368202 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrdtm"] Jul 20 16:46:33.374249 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:46:33.374214 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod946f07eb_462e_48d7_a469_692c829dfabf.slice/crio-00d10ace25856515094082a3f5928e859c3d7917ae69716c3f17ec49f24ede7e WatchSource:0}: Error finding container 00d10ace25856515094082a3f5928e859c3d7917ae69716c3f17ec49f24ede7e: Status 404 returned error can't find the container with id 00d10ace25856515094082a3f5928e859c3d7917ae69716c3f17ec49f24ede7e Jul 20 16:46:34.051908 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.051805 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tdgwh" event={"ID":"98ea33d6-9c67-4e13-842a-caffa292568c","Type":"ContainerStarted","Data":"9f552d24bbf2724050e558557c281dd9ac2ace007b94a7482651cb4f471d4439"} Jul 20 16:46:34.053177 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.053141 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrdtm" event={"ID":"946f07eb-462e-48d7-a469-692c829dfabf","Type":"ContainerStarted","Data":"00d10ace25856515094082a3f5928e859c3d7917ae69716c3f17ec49f24ede7e"} Jul 20 16:46:34.054948 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.054912 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2xvv9" event={"ID":"5b354580-bc22-4283-bbff-5e9e87fc026a","Type":"ContainerStarted","Data":"585cae5f047026075ebc5a5b7d85ffa8cc0432a544b704185f21c47fa1b99e8f"} Jul 20 16:46:34.057069 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.057045 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dwlfm" event={"ID":"1014a924-f279-4b0f-b362-4f39c0c03be0","Type":"ContainerStarted","Data":"1486265ebdf07f7b94aa616360a8ae3ddc219f6f2d821f970650b52a8755b983"} Jul 20 16:46:34.057179 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.057089 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dwlfm" event={"ID":"1014a924-f279-4b0f-b362-4f39c0c03be0","Type":"ContainerStarted","Data":"9173245678b9902726354cf1d1d2a08e9ebac54b526b460e798c938586496e8b"} Jul 20 16:46:34.057229 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.057206 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dwlfm" Jul 20 16:46:34.072459 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.072407 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2xvv9" podStartSLOduration=2.00106901 podStartE2EDuration="8.07238932s" totalImagesPullingTime="6.07132031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:34.070882986 +0000 UTC m=+58.856882077" watchObservedRunningTime="2026-07-20 16:46:34.07238932 +0000 UTC m=+58.858388378" Jul 20 16:46:34.086417 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.086364 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dwlfm" podStartSLOduration=2.34081596 podStartE2EDuration="8.086346088s" totalImagesPullingTime="5.745530128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:34.086311805 +0000 UTC m=+58.872310885" watchObservedRunningTime="2026-07-20 16:46:34.086346088 +0000 UTC m=+58.872345169" Jul 20 16:46:34.216458 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.216424 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-ddc59cf58-8smsw"] Jul 20 16:46:34.218586 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.218564 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.221087 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.220610 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kgxrj\"" Jul 20 16:46:34.221087 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.220857 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 20 16:46:34.221087 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.220872 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 20 16:46:34.221087 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.220948 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 20 16:46:34.221087 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.220957 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 20 16:46:34.221374 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.221141 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 20 16:46:34.221374 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.221267 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 20 16:46:34.221479 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.221398 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 20 16:46:34.229911 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.229891 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ddc59cf58-8smsw"] Jul 20 16:46:34.274931 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.274900 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/896660eb-da4e-4fb5-b0df-7bfa50ed6477-console-config\") pod \"console-ddc59cf58-8smsw\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.275109 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.274938 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/896660eb-da4e-4fb5-b0df-7bfa50ed6477-oauth-serving-cert\") pod \"console-ddc59cf58-8smsw\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.275109 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.274970 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8rqx\" (UniqueName: \"kubernetes.io/projected/896660eb-da4e-4fb5-b0df-7bfa50ed6477-kube-api-access-s8rqx\") pod \"console-ddc59cf58-8smsw\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.275109 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.275038 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/896660eb-da4e-4fb5-b0df-7bfa50ed6477-console-oauth-config\") pod \"console-ddc59cf58-8smsw\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.275109 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.275067 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/896660eb-da4e-4fb5-b0df-7bfa50ed6477-console-serving-cert\") pod \"console-ddc59cf58-8smsw\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.275109 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.275093 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/896660eb-da4e-4fb5-b0df-7bfa50ed6477-service-ca\") pod \"console-ddc59cf58-8smsw\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.376058 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.376025 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/896660eb-da4e-4fb5-b0df-7bfa50ed6477-console-config\") pod \"console-ddc59cf58-8smsw\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.376058 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.376067 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/896660eb-da4e-4fb5-b0df-7bfa50ed6477-oauth-serving-cert\") pod \"console-ddc59cf58-8smsw\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.376583 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.376096 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8rqx\" (UniqueName: \"kubernetes.io/projected/896660eb-da4e-4fb5-b0df-7bfa50ed6477-kube-api-access-s8rqx\") pod \"console-ddc59cf58-8smsw\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.376583 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.376137 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/896660eb-da4e-4fb5-b0df-7bfa50ed6477-console-oauth-config\") pod \"console-ddc59cf58-8smsw\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.376583 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.376169 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/896660eb-da4e-4fb5-b0df-7bfa50ed6477-console-serving-cert\") pod \"console-ddc59cf58-8smsw\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.376583 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.376205 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/896660eb-da4e-4fb5-b0df-7bfa50ed6477-service-ca\") pod \"console-ddc59cf58-8smsw\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.376953 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.376929 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/896660eb-da4e-4fb5-b0df-7bfa50ed6477-console-config\") pod \"console-ddc59cf58-8smsw\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.376953 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.376939 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/896660eb-da4e-4fb5-b0df-7bfa50ed6477-oauth-serving-cert\") pod \"console-ddc59cf58-8smsw\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.377070 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.377003 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/896660eb-da4e-4fb5-b0df-7bfa50ed6477-service-ca\") pod \"console-ddc59cf58-8smsw\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.379864 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.379819 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/896660eb-da4e-4fb5-b0df-7bfa50ed6477-console-oauth-config\") pod \"console-ddc59cf58-8smsw\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.380419 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.380375 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/896660eb-da4e-4fb5-b0df-7bfa50ed6477-console-serving-cert\") pod \"console-ddc59cf58-8smsw\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.390502 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.390470 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8rqx\" (UniqueName: \"kubernetes.io/projected/896660eb-da4e-4fb5-b0df-7bfa50ed6477-kube-api-access-s8rqx\") pod \"console-ddc59cf58-8smsw\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.531483 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.531442 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:34.704788 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:34.704741 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ddc59cf58-8smsw"] Jul 20 16:46:34.922877 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:46:34.922805 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod896660eb_da4e_4fb5_b0df_7bfa50ed6477.slice/crio-bf3b237796573ade86ca49750b59f939f5f47622986d085f212396cd04bd0cfa WatchSource:0}: Error finding container bf3b237796573ade86ca49750b59f939f5f47622986d085f212396cd04bd0cfa: Status 404 returned error can't find the container with id bf3b237796573ade86ca49750b59f939f5f47622986d085f212396cd04bd0cfa Jul 20 16:46:35.061379 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:35.061338 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ddc59cf58-8smsw" event={"ID":"896660eb-da4e-4fb5-b0df-7bfa50ed6477","Type":"ContainerStarted","Data":"bf3b237796573ade86ca49750b59f939f5f47622986d085f212396cd04bd0cfa"} Jul 20 16:46:36.983425 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:36.983387 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l5n5k" Jul 20 16:46:38.072645 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:38.072589 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tdgwh" event={"ID":"98ea33d6-9c67-4e13-842a-caffa292568c","Type":"ContainerStarted","Data":"2cb6501c539a9ab05d9c2d5b9a16d12f5f3c4ec4002dc96597a8c36c09948352"} Jul 20 16:46:38.074012 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:38.073990 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrdtm" event={"ID":"946f07eb-462e-48d7-a469-692c829dfabf","Type":"ContainerStarted","Data":"4b89657b20d0388fff791ba311f06472c3f405c420aa4699fc61b2ec3bc3c9d9"} Jul 20 16:46:38.074223 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:38.074202 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrdtm" Jul 20 16:46:38.079062 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:38.079031 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrdtm" Jul 20 16:46:38.089821 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:38.089748 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrdtm" podStartSLOduration=5.400602759 podStartE2EDuration="9.089733963s" totalImagesPullingTime="3.689131204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:38.089234928 +0000 UTC m=+62.875234007" watchObservedRunningTime="2026-07-20 16:46:38.089733963 +0000 UTC m=+62.875733042" Jul 20 16:46:38.858423 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:38.858389 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-777dm"] Jul 20 16:46:38.862726 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:38.862697 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-777dm" Jul 20 16:46:38.865981 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:38.865949 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 20 16:46:38.866117 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:38.866029 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 20 16:46:38.866271 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:38.866252 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-2jlns\"" Jul 20 16:46:38.866489 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:38.866467 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 20 16:46:38.867416 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:38.866680 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 20 16:46:38.867416 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:38.866904 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 20 16:46:38.871803 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:38.871564 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-777dm"] Jul 20 16:46:38.916619 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:38.916581 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19715257-abd0-4174-8253-48fecc5b79a9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-777dm\" (UID: \"19715257-abd0-4174-8253-48fecc5b79a9\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-777dm" Jul 20 16:46:38.916831 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:38.916658 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/19715257-abd0-4174-8253-48fecc5b79a9-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-777dm\" (UID: \"19715257-abd0-4174-8253-48fecc5b79a9\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-777dm" Jul 20 16:46:38.916831 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:38.916701 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19715257-abd0-4174-8253-48fecc5b79a9-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-777dm\" (UID: \"19715257-abd0-4174-8253-48fecc5b79a9\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-777dm" Jul 20 16:46:38.916831 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:38.916740 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ls8g\" (UniqueName: \"kubernetes.io/projected/19715257-abd0-4174-8253-48fecc5b79a9-kube-api-access-8ls8g\") pod \"prometheus-operator-5945b5cfd-777dm\" (UID: \"19715257-abd0-4174-8253-48fecc5b79a9\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-777dm" Jul 20 16:46:39.018082 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:39.018030 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ls8g\" (UniqueName: \"kubernetes.io/projected/19715257-abd0-4174-8253-48fecc5b79a9-kube-api-access-8ls8g\") pod \"prometheus-operator-5945b5cfd-777dm\" (UID: \"19715257-abd0-4174-8253-48fecc5b79a9\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-777dm" Jul 20 16:46:39.018267 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:39.018140 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19715257-abd0-4174-8253-48fecc5b79a9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-777dm\" (UID: \"19715257-abd0-4174-8253-48fecc5b79a9\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-777dm" Jul 20 16:46:39.018267 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:39.018169 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/19715257-abd0-4174-8253-48fecc5b79a9-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-777dm\" (UID: \"19715257-abd0-4174-8253-48fecc5b79a9\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-777dm" Jul 20 16:46:39.018267 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:39.018193 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19715257-abd0-4174-8253-48fecc5b79a9-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-777dm\" (UID: \"19715257-abd0-4174-8253-48fecc5b79a9\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-777dm" Jul 20 16:46:39.018901 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:39.018872 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19715257-abd0-4174-8253-48fecc5b79a9-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-777dm\" (UID: \"19715257-abd0-4174-8253-48fecc5b79a9\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-777dm" Jul 20 16:46:39.021986 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:39.021959 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/19715257-abd0-4174-8253-48fecc5b79a9-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-777dm\" (UID: \"19715257-abd0-4174-8253-48fecc5b79a9\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-777dm" Jul 20 16:46:39.027199 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:39.027166 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ls8g\" (UniqueName: \"kubernetes.io/projected/19715257-abd0-4174-8253-48fecc5b79a9-kube-api-access-8ls8g\") pod \"prometheus-operator-5945b5cfd-777dm\" (UID: \"19715257-abd0-4174-8253-48fecc5b79a9\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-777dm" Jul 20 16:46:39.028214 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:39.028167 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19715257-abd0-4174-8253-48fecc5b79a9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-777dm\" (UID: \"19715257-abd0-4174-8253-48fecc5b79a9\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-777dm" Jul 20 16:46:39.175718 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:39.175638 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-777dm" Jul 20 16:46:39.456983 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:39.456850 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-777dm"] Jul 20 16:46:40.082337 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:40.082300 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-777dm" event={"ID":"19715257-abd0-4174-8253-48fecc5b79a9","Type":"ContainerStarted","Data":"53041f7e4fb555dcd73bea7e870d2eafd49393c9a3833b6f97b688b2ebbe3a22"} Jul 20 16:46:40.083942 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:40.083901 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ddc59cf58-8smsw" event={"ID":"896660eb-da4e-4fb5-b0df-7bfa50ed6477","Type":"ContainerStarted","Data":"81ebd5140b3085cf76f536171b223ade9193ff521322aa53a6493fac1ba3408d"} Jul 20 16:46:40.101528 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:40.101465 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-ddc59cf58-8smsw" podStartSLOduration=1.684952547 podStartE2EDuration="6.101445166s" totalImagesPullingTime="4.416492619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:40.100624772 +0000 UTC m=+64.886623852" watchObservedRunningTime="2026-07-20 16:46:40.101445166 +0000 UTC m=+64.887444242" Jul 20 16:46:40.431185 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:40.431096 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret\") pod \"global-pull-secret-syncer-t2zqd\" (UID: \"808cf239-1e7e-484f-89f2-e8e9ad673366\") " pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:40.433514 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:40.433486 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 16:46:40.443844 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:40.443816 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/808cf239-1e7e-484f-89f2-e8e9ad673366-original-pull-secret\") pod \"global-pull-secret-syncer-t2zqd\" (UID: \"808cf239-1e7e-484f-89f2-e8e9ad673366\") " pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:40.532275 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:40.532230 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs\") pod \"network-metrics-daemon-6xs67\" (UID: \"b18dfe54-9dfb-4d77-b2f5-5fe6837623d5\") " pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:40.534504 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:40.534480 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 16:46:40.545888 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:40.545861 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b18dfe54-9dfb-4d77-b2f5-5fe6837623d5-metrics-certs\") pod \"network-metrics-daemon-6xs67\" (UID: \"b18dfe54-9dfb-4d77-b2f5-5fe6837623d5\") " pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:40.621841 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:40.621807 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2sqpl\"" Jul 20 16:46:40.630567 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:40.630531 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6xs67" Jul 20 16:46:40.633075 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:40.632974 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k952p\" (UniqueName: \"kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p\") pod \"network-check-target-qfqcm\" (UID: \"9ef980cf-2026-4f9a-9e98-b6e02d793100\") " pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:40.633871 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:40.633849 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2zqd" Jul 20 16:46:40.634854 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:40.634832 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 16:46:40.645458 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:40.645440 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 16:46:40.656597 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:40.656562 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k952p\" (UniqueName: \"kubernetes.io/projected/9ef980cf-2026-4f9a-9e98-b6e02d793100-kube-api-access-k952p\") pod \"network-check-target-qfqcm\" (UID: \"9ef980cf-2026-4f9a-9e98-b6e02d793100\") " pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:40.941230 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:40.941196 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9vk4l\"" Jul 20 16:46:40.949557 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:40.949525 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:41.468154 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:41.468121 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qfqcm"] Jul 20 16:46:41.472587 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:46:41.472557 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ef980cf_2026_4f9a_9e98_b6e02d793100.slice/crio-2de79e66d107355718df603c0bd84795c631ef4e2acfa280ddd0cecfd1d1171e WatchSource:0}: Error finding container 2de79e66d107355718df603c0bd84795c631ef4e2acfa280ddd0cecfd1d1171e: Status 404 returned error can't find the container with id 2de79e66d107355718df603c0bd84795c631ef4e2acfa280ddd0cecfd1d1171e Jul 20 16:46:41.484280 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:41.484250 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t2zqd"] Jul 20 16:46:41.487525 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:46:41.487502 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod808cf239_1e7e_484f_89f2_e8e9ad673366.slice/crio-9d84bfedc8942f707b44fae674d2768380dd04eba55f9a581c8f9a9ec7cf221f WatchSource:0}: Error finding container 9d84bfedc8942f707b44fae674d2768380dd04eba55f9a581c8f9a9ec7cf221f: Status 404 returned error can't find the container with id 9d84bfedc8942f707b44fae674d2768380dd04eba55f9a581c8f9a9ec7cf221f Jul 20 16:46:41.503382 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:41.503344 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6xs67"] Jul 20 16:46:41.506612 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:46:41.506586 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb18dfe54_9dfb_4d77_b2f5_5fe6837623d5.slice/crio-00f632db795aa5b9f95098c09d89f5aa757e253f34be3ba13c38891786866379 WatchSource:0}: Error finding container 00f632db795aa5b9f95098c09d89f5aa757e253f34be3ba13c38891786866379: Status 404 returned error can't find the container with id 00f632db795aa5b9f95098c09d89f5aa757e253f34be3ba13c38891786866379 Jul 20 16:46:42.095179 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:42.095142 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qfqcm" event={"ID":"9ef980cf-2026-4f9a-9e98-b6e02d793100","Type":"ContainerStarted","Data":"2de79e66d107355718df603c0bd84795c631ef4e2acfa280ddd0cecfd1d1171e"} Jul 20 16:46:42.096976 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:42.096943 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t2zqd" event={"ID":"808cf239-1e7e-484f-89f2-e8e9ad673366","Type":"ContainerStarted","Data":"9d84bfedc8942f707b44fae674d2768380dd04eba55f9a581c8f9a9ec7cf221f"} Jul 20 16:46:42.098535 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:42.098508 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6xs67" event={"ID":"b18dfe54-9dfb-4d77-b2f5-5fe6837623d5","Type":"ContainerStarted","Data":"00f632db795aa5b9f95098c09d89f5aa757e253f34be3ba13c38891786866379"} Jul 20 16:46:42.102260 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:42.101530 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tdgwh" event={"ID":"98ea33d6-9c67-4e13-842a-caffa292568c","Type":"ContainerStarted","Data":"e9c7a576584011fc1af930e2eed5008aa420679a8e9943a29f6837b2894ac3ac"} Jul 20 16:46:42.119606 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:42.119541 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tdgwh" podStartSLOduration=2.186190895 podStartE2EDuration="16.119521204s" totalImagesPullingTime="13.933330309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:46:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:42.119095006 +0000 UTC m=+66.905094085" watchObservedRunningTime="2026-07-20 16:46:42.119521204 +0000 UTC m=+66.905520285" Jul 20 16:46:43.108065 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:43.108022 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6xs67" event={"ID":"b18dfe54-9dfb-4d77-b2f5-5fe6837623d5","Type":"ContainerStarted","Data":"b1aed279611da9d6165278f7eb190d43accc6f079467f03ad43bdc28256949d8"} Jul 20 16:46:43.108512 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:43.108088 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6xs67" event={"ID":"b18dfe54-9dfb-4d77-b2f5-5fe6837623d5","Type":"ContainerStarted","Data":"c56a9b1142fffeceda42177b1b0da6bd12b387374559841918cb33181383de89"} Jul 20 16:46:43.110441 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:43.110411 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-777dm" event={"ID":"19715257-abd0-4174-8253-48fecc5b79a9","Type":"ContainerStarted","Data":"d33daab35c4d7a2a7093f023ebda808ad3be14e4627d7e617bbf1e9714a235b7"} Jul 20 16:46:43.110574 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:43.110471 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-777dm" event={"ID":"19715257-abd0-4174-8253-48fecc5b79a9","Type":"ContainerStarted","Data":"79ce1e1926d7a2624b38dd366bd96d5a03805aa448ecbe87d8410d9869de7c9d"} Jul 20 16:46:43.141159 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:43.140476 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-777dm" podStartSLOduration=2.037991733 podStartE2EDuration="5.140453981s" totalImagesPullingTime="3.102462248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:43.139624463 +0000 UTC m=+67.925623542" watchObservedRunningTime="2026-07-20 16:46:43.140453981 +0000 UTC m=+67.926453061" Jul 20 16:46:43.141159 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:43.140935 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6xs67" podStartSLOduration=66.079697306 podStartE2EDuration="1m7.140926267s" totalImagesPullingTime="1.061228961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:43.123867076 +0000 UTC m=+67.909866148" watchObservedRunningTime="2026-07-20 16:46:43.140926267 +0000 UTC m=+67.926925347" Jul 20 16:46:44.063693 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:44.063660 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dwlfm" Jul 20 16:46:44.532569 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:44.532524 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:44.533050 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:44.532697 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:44.538165 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:44.538134 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:45.124326 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.124292 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:46:45.214111 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.214078 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f5d69d4fd-mbdvm"] Jul 20 16:46:45.236168 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.236138 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f5d69d4fd-mbdvm"] Jul 20 16:46:45.236315 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.236267 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.243456 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.243425 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 20 16:46:45.245606 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.245585 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9"] Jul 20 16:46:45.266222 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.266194 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9"] Jul 20 16:46:45.266367 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.266324 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" Jul 20 16:46:45.269090 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.269070 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 20 16:46:45.269090 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.269082 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 20 16:46:45.269409 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.269383 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-9kmgv\"" Jul 20 16:46:45.270936 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.270910 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9w4x\" (UniqueName: \"kubernetes.io/projected/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-kube-api-access-r9w4x\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.271061 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.270959 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-trusted-ca-bundle\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.271061 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.270982 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-service-ca\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.271155 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.271074 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-console-serving-cert\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.271155 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.271113 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-console-oauth-config\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.271213 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.271152 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-console-config\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.271213 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.271179 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-oauth-serving-cert\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.278449 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.278427 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-n8f49"] Jul 20 16:46:45.300727 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.300364 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jr2tv"] Jul 20 16:46:45.300727 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.300580 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.303048 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.303023 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 20 16:46:45.304017 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.303802 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 20 16:46:45.305043 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.304826 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qfwbl\"" Jul 20 16:46:45.305043 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.304853 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 20 16:46:45.315743 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.315683 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-n8f49"] Jul 20 16:46:45.315908 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.315876 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.317957 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.317938 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2tw6m\"" Jul 20 16:46:45.318069 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.317997 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 20 16:46:45.318118 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.317941 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 20 16:46:45.318206 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.318192 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 20 16:46:45.372706 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.372222 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-console-oauth-config\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.372706 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.372286 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/125e8fb6-3cf7-4bf6-ba49-b02a6f580e44-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-n8f49\" (UID: \"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.372706 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.372310 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-console-config\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.372706 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.372335 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-oauth-serving-cert\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.372706 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.372363 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prfjk\" (UniqueName: \"kubernetes.io/projected/e2c1840a-8c58-4136-8479-c700fb9ccd2c-kube-api-access-prfjk\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.373487 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373327 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/125e8fb6-3cf7-4bf6-ba49-b02a6f580e44-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-n8f49\" (UID: \"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.373487 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373398 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e2c1840a-8c58-4136-8479-c700fb9ccd2c-node-exporter-accelerators-collector-config\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.373487 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373434 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c716602-e8f7-4a96-ba67-b64a6126a0a2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-gkfd9\" (UID: \"6c716602-e8f7-4a96-ba67-b64a6126a0a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" Jul 20 16:46:45.373487 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373463 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2c1840a-8c58-4136-8479-c700fb9ccd2c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.373746 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373538 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2c1840a-8c58-4136-8479-c700fb9ccd2c-metrics-client-ca\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.373746 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373577 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/125e8fb6-3cf7-4bf6-ba49-b02a6f580e44-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-n8f49\" (UID: \"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.373746 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373612 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9w4x\" (UniqueName: \"kubernetes.io/projected/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-kube-api-access-r9w4x\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.373746 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373649 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c716602-e8f7-4a96-ba67-b64a6126a0a2-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-gkfd9\" (UID: \"6c716602-e8f7-4a96-ba67-b64a6126a0a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" Jul 20 16:46:45.373746 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373696 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9fxz\" (UniqueName: \"kubernetes.io/projected/125e8fb6-3cf7-4bf6-ba49-b02a6f580e44-kube-api-access-p9fxz\") pod \"kube-state-metrics-56c5594674-n8f49\" (UID: \"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.373746 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373730 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-trusted-ca-bundle\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.374074 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373755 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-service-ca\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.374074 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373812 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lscxj\" (UniqueName: \"kubernetes.io/projected/6c716602-e8f7-4a96-ba67-b64a6126a0a2-kube-api-access-lscxj\") pod \"openshift-state-metrics-79f6958cdf-gkfd9\" (UID: \"6c716602-e8f7-4a96-ba67-b64a6126a0a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" Jul 20 16:46:45.374074 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373840 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e2c1840a-8c58-4136-8479-c700fb9ccd2c-root\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.374074 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373863 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/125e8fb6-3cf7-4bf6-ba49-b02a6f580e44-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-n8f49\" (UID: \"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.374074 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373892 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e2c1840a-8c58-4136-8479-c700fb9ccd2c-node-exporter-textfile\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.374074 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373913 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e2c1840a-8c58-4136-8479-c700fb9ccd2c-node-exporter-tls\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.374074 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373939 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2c1840a-8c58-4136-8479-c700fb9ccd2c-sys\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.374074 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373955 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e2c1840a-8c58-4136-8479-c700fb9ccd2c-node-exporter-wtmp\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.374074 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373977 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-console-serving-cert\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.374074 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.373991 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-console-config\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.374074 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.374001 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c716602-e8f7-4a96-ba67-b64a6126a0a2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-gkfd9\" (UID: \"6c716602-e8f7-4a96-ba67-b64a6126a0a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" Jul 20 16:46:45.374074 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.374023 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-oauth-serving-cert\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.374074 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.374018 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/125e8fb6-3cf7-4bf6-ba49-b02a6f580e44-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-n8f49\" (UID: \"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.374741 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.374678 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-service-ca\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.374912 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.374894 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-trusted-ca-bundle\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.376430 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.376407 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-console-oauth-config\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.376718 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.376696 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-console-serving-cert\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.382734 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.382336 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9w4x\" (UniqueName: \"kubernetes.io/projected/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-kube-api-access-r9w4x\") pod \"console-7f5d69d4fd-mbdvm\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.475123 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475066 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prfjk\" (UniqueName: \"kubernetes.io/projected/e2c1840a-8c58-4136-8479-c700fb9ccd2c-kube-api-access-prfjk\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.475123 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475128 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/125e8fb6-3cf7-4bf6-ba49-b02a6f580e44-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-n8f49\" (UID: \"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.475359 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475150 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e2c1840a-8c58-4136-8479-c700fb9ccd2c-node-exporter-accelerators-collector-config\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.475359 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475172 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c716602-e8f7-4a96-ba67-b64a6126a0a2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-gkfd9\" (UID: \"6c716602-e8f7-4a96-ba67-b64a6126a0a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" Jul 20 16:46:45.475359 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475189 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2c1840a-8c58-4136-8479-c700fb9ccd2c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.475359 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475213 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2c1840a-8c58-4136-8479-c700fb9ccd2c-metrics-client-ca\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.475359 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475241 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/125e8fb6-3cf7-4bf6-ba49-b02a6f580e44-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-n8f49\" (UID: \"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.475359 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475269 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c716602-e8f7-4a96-ba67-b64a6126a0a2-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-gkfd9\" (UID: \"6c716602-e8f7-4a96-ba67-b64a6126a0a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" Jul 20 16:46:45.475359 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475295 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9fxz\" (UniqueName: \"kubernetes.io/projected/125e8fb6-3cf7-4bf6-ba49-b02a6f580e44-kube-api-access-p9fxz\") pod \"kube-state-metrics-56c5594674-n8f49\" (UID: \"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.475359 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475339 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lscxj\" (UniqueName: \"kubernetes.io/projected/6c716602-e8f7-4a96-ba67-b64a6126a0a2-kube-api-access-lscxj\") pod \"openshift-state-metrics-79f6958cdf-gkfd9\" (UID: \"6c716602-e8f7-4a96-ba67-b64a6126a0a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" Jul 20 16:46:45.475359 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475361 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e2c1840a-8c58-4136-8479-c700fb9ccd2c-root\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.475841 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475385 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/125e8fb6-3cf7-4bf6-ba49-b02a6f580e44-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-n8f49\" (UID: \"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.475841 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475413 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e2c1840a-8c58-4136-8479-c700fb9ccd2c-node-exporter-textfile\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.475841 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475466 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e2c1840a-8c58-4136-8479-c700fb9ccd2c-node-exporter-tls\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.475841 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475508 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2c1840a-8c58-4136-8479-c700fb9ccd2c-sys\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.475841 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475533 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e2c1840a-8c58-4136-8479-c700fb9ccd2c-node-exporter-wtmp\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.475841 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475559 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c716602-e8f7-4a96-ba67-b64a6126a0a2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-gkfd9\" (UID: \"6c716602-e8f7-4a96-ba67-b64a6126a0a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" Jul 20 16:46:45.475841 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475583 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/125e8fb6-3cf7-4bf6-ba49-b02a6f580e44-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-n8f49\" (UID: \"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.475841 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475633 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/125e8fb6-3cf7-4bf6-ba49-b02a6f580e44-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-n8f49\" (UID: \"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.476228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475842 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e2c1840a-8c58-4136-8479-c700fb9ccd2c-node-exporter-accelerators-collector-config\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.476228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475865 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/125e8fb6-3cf7-4bf6-ba49-b02a6f580e44-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-n8f49\" (UID: \"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.476228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.475941 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e2c1840a-8c58-4136-8479-c700fb9ccd2c-root\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.476373 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.476246 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2c1840a-8c58-4136-8479-c700fb9ccd2c-sys\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.476373 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.476297 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/125e8fb6-3cf7-4bf6-ba49-b02a6f580e44-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-n8f49\" (UID: \"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.476525 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.476420 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e2c1840a-8c58-4136-8479-c700fb9ccd2c-node-exporter-wtmp\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.477764 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.476667 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c716602-e8f7-4a96-ba67-b64a6126a0a2-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-gkfd9\" (UID: \"6c716602-e8f7-4a96-ba67-b64a6126a0a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" Jul 20 16:46:45.477764 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.476870 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e2c1840a-8c58-4136-8479-c700fb9ccd2c-node-exporter-textfile\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.477764 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.477135 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2c1840a-8c58-4136-8479-c700fb9ccd2c-metrics-client-ca\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.479371 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.479335 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/125e8fb6-3cf7-4bf6-ba49-b02a6f580e44-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-n8f49\" (UID: \"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.480260 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.480235 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2c1840a-8c58-4136-8479-c700fb9ccd2c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.480665 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.480438 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e2c1840a-8c58-4136-8479-c700fb9ccd2c-node-exporter-tls\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.480751 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.480419 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/125e8fb6-3cf7-4bf6-ba49-b02a6f580e44-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-n8f49\" (UID: \"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.481974 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.481951 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c716602-e8f7-4a96-ba67-b64a6126a0a2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-gkfd9\" (UID: \"6c716602-e8f7-4a96-ba67-b64a6126a0a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" Jul 20 16:46:45.483853 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.483828 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/125e8fb6-3cf7-4bf6-ba49-b02a6f580e44-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-n8f49\" (UID: \"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.485195 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.485176 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c716602-e8f7-4a96-ba67-b64a6126a0a2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-gkfd9\" (UID: \"6c716602-e8f7-4a96-ba67-b64a6126a0a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" Jul 20 16:46:45.488238 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.488219 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prfjk\" (UniqueName: \"kubernetes.io/projected/e2c1840a-8c58-4136-8479-c700fb9ccd2c-kube-api-access-prfjk\") pod \"node-exporter-jr2tv\" (UID: \"e2c1840a-8c58-4136-8479-c700fb9ccd2c\") " pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:45.494152 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.494131 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lscxj\" (UniqueName: \"kubernetes.io/projected/6c716602-e8f7-4a96-ba67-b64a6126a0a2-kube-api-access-lscxj\") pod \"openshift-state-metrics-79f6958cdf-gkfd9\" (UID: \"6c716602-e8f7-4a96-ba67-b64a6126a0a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" Jul 20 16:46:45.494874 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.494850 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9fxz\" (UniqueName: \"kubernetes.io/projected/125e8fb6-3cf7-4bf6-ba49-b02a6f580e44-kube-api-access-p9fxz\") pod \"kube-state-metrics-56c5594674-n8f49\" (UID: \"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.548690 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.548416 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:45.576632 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.576601 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" Jul 20 16:46:45.613732 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.613697 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" Jul 20 16:46:45.625809 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:45.625741 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jr2tv" Jul 20 16:46:46.281686 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.281655 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 16:46:46.284408 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.284377 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.286467 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.286442 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 20 16:46:46.286617 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.286587 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 20 16:46:46.286905 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.286874 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 20 16:46:46.286905 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.286885 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 20 16:46:46.287058 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.286913 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 20 16:46:46.287058 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.286932 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 20 16:46:46.287350 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.287326 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 20 16:46:46.287427 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.287379 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 20 16:46:46.287491 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.287439 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 20 16:46:46.287582 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.287561 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bvm75\"" Jul 20 16:46:46.301041 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.300192 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 16:46:46.383486 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.383450 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c49477a4-db75-4ac8-80fe-159ce55be73c-config-out\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.383676 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.383505 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c49477a4-db75-4ac8-80fe-159ce55be73c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.383676 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.383537 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.383676 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.383589 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.383849 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.383706 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c49477a4-db75-4ac8-80fe-159ce55be73c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.383849 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.383756 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-config-volume\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.383849 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.383794 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.383849 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.383825 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.384019 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.383883 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c49477a4-db75-4ac8-80fe-159ce55be73c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.384019 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.383920 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c49477a4-db75-4ac8-80fe-159ce55be73c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.384019 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.383944 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv5r6\" (UniqueName: \"kubernetes.io/projected/c49477a4-db75-4ac8-80fe-159ce55be73c-kube-api-access-vv5r6\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.384019 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.383967 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.384019 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.383990 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-web-config\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.468271 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:46:46.468174 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2c1840a_8c58_4136_8479_c700fb9ccd2c.slice/crio-8e24cd015e9349b1b4d8be2f01f243c87ad24624ced70f9db0c3bf5620ad5b52 WatchSource:0}: Error finding container 8e24cd015e9349b1b4d8be2f01f243c87ad24624ced70f9db0c3bf5620ad5b52: Status 404 returned error can't find the container with id 8e24cd015e9349b1b4d8be2f01f243c87ad24624ced70f9db0c3bf5620ad5b52 Jul 20 16:46:46.487669 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.485557 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c49477a4-db75-4ac8-80fe-159ce55be73c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.487669 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.485607 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.487669 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.485640 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.487669 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.485669 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c49477a4-db75-4ac8-80fe-159ce55be73c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.487669 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.485695 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-config-volume\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.487669 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.485721 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.487669 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.485762 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.487669 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.485814 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c49477a4-db75-4ac8-80fe-159ce55be73c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.487669 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.485843 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c49477a4-db75-4ac8-80fe-159ce55be73c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.487669 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.485873 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv5r6\" (UniqueName: \"kubernetes.io/projected/c49477a4-db75-4ac8-80fe-159ce55be73c-kube-api-access-vv5r6\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.487669 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.485913 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.487669 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.485940 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-web-config\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.487669 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.485973 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c49477a4-db75-4ac8-80fe-159ce55be73c-config-out\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.488799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.488346 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c49477a4-db75-4ac8-80fe-159ce55be73c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.489762 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.489647 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c49477a4-db75-4ac8-80fe-159ce55be73c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.490679 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.490609 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c49477a4-db75-4ac8-80fe-159ce55be73c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.497031 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.493009 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.497674 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.497622 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv5r6\" (UniqueName: \"kubernetes.io/projected/c49477a4-db75-4ac8-80fe-159ce55be73c-kube-api-access-vv5r6\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.501464 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.501437 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-config-volume\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.502101 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.502075 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.502231 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.502160 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.502451 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.502429 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c49477a4-db75-4ac8-80fe-159ce55be73c-config-out\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.502814 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.502748 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c49477a4-db75-4ac8-80fe-159ce55be73c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.503527 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.503221 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.504275 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.504252 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-web-config\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.509653 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.509307 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.595858 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.595831 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:46:46.642305 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.642187 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f5d69d4fd-mbdvm"] Jul 20 16:46:46.670692 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.670067 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9"] Jul 20 16:46:46.674002 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:46:46.673965 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c716602_e8f7_4a96_ba67_b64a6126a0a2.slice/crio-78da57397322500e6e8d1d96e61322649e7bbe8cc753628ed29b27e68ea04e5c WatchSource:0}: Error finding container 78da57397322500e6e8d1d96e61322649e7bbe8cc753628ed29b27e68ea04e5c: Status 404 returned error can't find the container with id 78da57397322500e6e8d1d96e61322649e7bbe8cc753628ed29b27e68ea04e5c Jul 20 16:46:46.779583 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.779553 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 16:46:46.782657 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:46:46.782610 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc49477a4_db75_4ac8_80fe_159ce55be73c.slice/crio-2062556e02b331162553cd8e427da451de841bf59c459e13bd403e7419d068c7 WatchSource:0}: Error finding container 2062556e02b331162553cd8e427da451de841bf59c459e13bd403e7419d068c7: Status 404 returned error can't find the container with id 2062556e02b331162553cd8e427da451de841bf59c459e13bd403e7419d068c7 Jul 20 16:46:46.873440 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:46.873393 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-n8f49"] Jul 20 16:46:46.876981 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:46:46.876952 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod125e8fb6_3cf7_4bf6_ba49_b02a6f580e44.slice/crio-1d9b8d7350fb282831a84a3f2af1b6b5b63347768a0e79270c83bfa1d448aefa WatchSource:0}: Error finding container 1d9b8d7350fb282831a84a3f2af1b6b5b63347768a0e79270c83bfa1d448aefa: Status 404 returned error can't find the container with id 1d9b8d7350fb282831a84a3f2af1b6b5b63347768a0e79270c83bfa1d448aefa Jul 20 16:46:47.131826 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:47.131790 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qfqcm" event={"ID":"9ef980cf-2026-4f9a-9e98-b6e02d793100","Type":"ContainerStarted","Data":"c5d4edda2a42b7c550792565a48acd0c702e771c1544bddc4449748ee51051e4"} Jul 20 16:46:47.132014 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:47.131920 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:46:47.134205 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:47.134102 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t2zqd" event={"ID":"808cf239-1e7e-484f-89f2-e8e9ad673366","Type":"ContainerStarted","Data":"0621fbd7217721c2a4ffd808c9cd6eea969b5c3c1b8bcc87223dfb01a73fa6a7"} Jul 20 16:46:47.135711 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:47.135688 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c49477a4-db75-4ac8-80fe-159ce55be73c","Type":"ContainerStarted","Data":"2062556e02b331162553cd8e427da451de841bf59c459e13bd403e7419d068c7"} Jul 20 16:46:47.137719 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:47.137699 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" event={"ID":"6c716602-e8f7-4a96-ba67-b64a6126a0a2","Type":"ContainerStarted","Data":"e3382854bd106732751219539672d98a5f696dcfc9ad8b7d9149413394354fdb"} Jul 20 16:46:47.137857 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:47.137750 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" event={"ID":"6c716602-e8f7-4a96-ba67-b64a6126a0a2","Type":"ContainerStarted","Data":"290d4620e837d422e6cf742567592513945dc4f03a3e4b33e5802445c851b4d8"} Jul 20 16:46:47.137857 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:47.137798 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" event={"ID":"6c716602-e8f7-4a96-ba67-b64a6126a0a2","Type":"ContainerStarted","Data":"78da57397322500e6e8d1d96e61322649e7bbe8cc753628ed29b27e68ea04e5c"} Jul 20 16:46:47.139806 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:47.139591 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5d69d4fd-mbdvm" event={"ID":"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd","Type":"ContainerStarted","Data":"a137923e465026d030ad0f97cc127529988c4d806b432f55f7319e95279372a9"} Jul 20 16:46:47.139806 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:47.139634 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5d69d4fd-mbdvm" event={"ID":"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd","Type":"ContainerStarted","Data":"6ece1d46c56d5832589f7cf0eb362058908c2750ad2ba5099f3e784fa4962050"} Jul 20 16:46:47.141571 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:47.141517 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" event={"ID":"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44","Type":"ContainerStarted","Data":"1d9b8d7350fb282831a84a3f2af1b6b5b63347768a0e79270c83bfa1d448aefa"} Jul 20 16:46:47.142990 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:47.142967 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jr2tv" event={"ID":"e2c1840a-8c58-4136-8479-c700fb9ccd2c","Type":"ContainerStarted","Data":"8e24cd015e9349b1b4d8be2f01f243c87ad24624ced70f9db0c3bf5620ad5b52"} Jul 20 16:46:47.147299 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:47.146845 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qfqcm" podStartSLOduration=66.151816245 podStartE2EDuration="1m11.146830098s" totalImagesPullingTime="4.995013853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:47.146227671 +0000 UTC m=+71.932226750" watchObservedRunningTime="2026-07-20 16:46:47.146830098 +0000 UTC m=+71.932829181" Jul 20 16:46:47.163927 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:47.163878 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f5d69d4fd-mbdvm" podStartSLOduration=2.163864166 podStartE2EDuration="2.163864166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:47.162346452 +0000 UTC m=+71.948345533" watchObservedRunningTime="2026-07-20 16:46:47.163864166 +0000 UTC m=+71.949863245" Jul 20 16:46:48.151429 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.151396 2566 generic.go:356] "Generic (PLEG): container finished" podID="e2c1840a-8c58-4136-8479-c700fb9ccd2c" containerID="c8fc53992edf516ee4171b4d8d1b393252551501a7ce50329d086eadbe2f96c7" exitCode=0 Jul 20 16:46:48.151875 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.151672 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jr2tv" event={"ID":"e2c1840a-8c58-4136-8479-c700fb9ccd2c","Type":"ContainerDied","Data":"c8fc53992edf516ee4171b4d8d1b393252551501a7ce50329d086eadbe2f96c7"} Jul 20 16:46:48.172239 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.172187 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-t2zqd" podStartSLOduration=67.182671603 podStartE2EDuration="1m12.17216868s" totalImagesPullingTime="4.989497077s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:47.175048269 +0000 UTC m=+71.961047352" watchObservedRunningTime="2026-07-20 16:46:48.17216868 +0000 UTC m=+72.958167760" Jul 20 16:46:48.193048 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.193015 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-fbb45cdcd-dz559"] Jul 20 16:46:48.196572 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.196553 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.199024 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.199005 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 20 16:46:48.199175 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.199048 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 20 16:46:48.199337 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.199309 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ljcqr\"" Jul 20 16:46:48.199444 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.199046 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 20 16:46:48.199444 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.199437 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6jnnbucf1039h\"" Jul 20 16:46:48.199530 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.199474 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 20 16:46:48.200350 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.199660 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 20 16:46:48.213942 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.213917 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-fbb45cdcd-dz559"] Jul 20 16:46:48.302323 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.302289 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8d7b1577-accd-455c-8511-9198592e5c51-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.302485 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.302336 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8zzp\" (UniqueName: \"kubernetes.io/projected/8d7b1577-accd-455c-8511-9198592e5c51-kube-api-access-z8zzp\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.302485 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.302364 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8d7b1577-accd-455c-8511-9198592e5c51-secret-thanos-querier-tls\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.302485 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.302425 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d7b1577-accd-455c-8511-9198592e5c51-metrics-client-ca\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.302613 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.302493 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8d7b1577-accd-455c-8511-9198592e5c51-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.302613 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.302542 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8d7b1577-accd-455c-8511-9198592e5c51-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.302613 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.302582 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8d7b1577-accd-455c-8511-9198592e5c51-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.302722 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.302610 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8d7b1577-accd-455c-8511-9198592e5c51-secret-grpc-tls\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.403505 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.403478 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8d7b1577-accd-455c-8511-9198592e5c51-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.403590 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.403522 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8d7b1577-accd-455c-8511-9198592e5c51-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.403590 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.403545 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8d7b1577-accd-455c-8511-9198592e5c51-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.403590 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.403569 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8d7b1577-accd-455c-8511-9198592e5c51-secret-grpc-tls\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.403839 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.403610 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8d7b1577-accd-455c-8511-9198592e5c51-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.403839 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.403641 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8zzp\" (UniqueName: \"kubernetes.io/projected/8d7b1577-accd-455c-8511-9198592e5c51-kube-api-access-z8zzp\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.403839 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.403672 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8d7b1577-accd-455c-8511-9198592e5c51-secret-thanos-querier-tls\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.403839 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.403717 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d7b1577-accd-455c-8511-9198592e5c51-metrics-client-ca\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.404454 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.404425 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d7b1577-accd-455c-8511-9198592e5c51-metrics-client-ca\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.406841 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.406750 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8d7b1577-accd-455c-8511-9198592e5c51-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.407210 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.407155 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8d7b1577-accd-455c-8511-9198592e5c51-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.407317 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.407256 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8d7b1577-accd-455c-8511-9198592e5c51-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.407627 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.407599 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8d7b1577-accd-455c-8511-9198592e5c51-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.408029 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.408001 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8d7b1577-accd-455c-8511-9198592e5c51-secret-thanos-querier-tls\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.408298 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.408275 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8d7b1577-accd-455c-8511-9198592e5c51-secret-grpc-tls\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.411096 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.411077 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8zzp\" (UniqueName: \"kubernetes.io/projected/8d7b1577-accd-455c-8511-9198592e5c51-kube-api-access-z8zzp\") pod \"thanos-querier-fbb45cdcd-dz559\" (UID: \"8d7b1577-accd-455c-8511-9198592e5c51\") " pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.513655 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.513630 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:48.731127 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:48.731080 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-fbb45cdcd-dz559"] Jul 20 16:46:48.735112 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:46:48.735070 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d7b1577_accd_455c_8511_9198592e5c51.slice/crio-b0fe4e05d7ed8925f901d5f337a3317319c3a8ecd64695f270475856e11dfae0 WatchSource:0}: Error finding container b0fe4e05d7ed8925f901d5f337a3317319c3a8ecd64695f270475856e11dfae0: Status 404 returned error can't find the container with id b0fe4e05d7ed8925f901d5f337a3317319c3a8ecd64695f270475856e11dfae0 Jul 20 16:46:49.157845 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.157809 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" event={"ID":"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44","Type":"ContainerStarted","Data":"36d0151a51f986ca57a110e7a28efebde3917091492f1c7f903340aff1c828a9"} Jul 20 16:46:49.158269 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.157884 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" event={"ID":"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44","Type":"ContainerStarted","Data":"a81b2d616ca5b17a65c66f09b05500dfdfb70723f19739a51de19bd238ca3dbe"} Jul 20 16:46:49.158269 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.157917 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" event={"ID":"125e8fb6-3cf7-4bf6-ba49-b02a6f580e44","Type":"ContainerStarted","Data":"be7e77309fe8ef2f82e3d77f8356e28df8cc7a599f8621381b7ef8b497fd56c8"} Jul 20 16:46:49.161038 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.160295 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jr2tv" event={"ID":"e2c1840a-8c58-4136-8479-c700fb9ccd2c","Type":"ContainerStarted","Data":"eba002fe163a7ffe63885688e4ce76ba6c7feff56ae3b26efc9d9f51d0714840"} Jul 20 16:46:49.161038 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.160354 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jr2tv" event={"ID":"e2c1840a-8c58-4136-8479-c700fb9ccd2c","Type":"ContainerStarted","Data":"c15895be1a260e3cf361a8b25d1a4877d473de9a02346d1db1917a35508ad226"} Jul 20 16:46:49.161702 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.161663 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" event={"ID":"8d7b1577-accd-455c-8511-9198592e5c51","Type":"ContainerStarted","Data":"b0fe4e05d7ed8925f901d5f337a3317319c3a8ecd64695f270475856e11dfae0"} Jul 20 16:46:49.174589 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.174112 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-n8f49" podStartSLOduration=2.647017265 podStartE2EDuration="4.174091459s" totalImagesPullingTime="1.527074194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:49.173839924 +0000 UTC m=+73.959839004" watchObservedRunningTime="2026-07-20 16:46:49.174091459 +0000 UTC m=+73.960090539" Jul 20 16:46:49.192399 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.192353 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jr2tv" podStartSLOduration=3.294749384 podStartE2EDuration="4.192336064s" totalImagesPullingTime="825.463918ms" totalInitContainerRuntime="72.122762ms" isStatefulPod=true podCreationTimestamp="2026-07-20 16:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:49.191231776 +0000 UTC m=+73.977230895" watchObservedRunningTime="2026-07-20 16:46:49.192336064 +0000 UTC m=+73.978335143" Jul 20 16:46:49.643738 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.643672 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-79fd676bf4-5kjr6"] Jul 20 16:46:49.645641 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.645622 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.648429 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.648403 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 20 16:46:49.648429 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.648422 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2r4h21kelfkkn\"" Jul 20 16:46:49.648616 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.648433 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 20 16:46:49.648616 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.648470 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 20 16:46:49.648616 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.648501 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 20 16:46:49.648616 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.648406 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-8zhd5\"" Jul 20 16:46:49.655993 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.655969 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79fd676bf4-5kjr6"] Jul 20 16:46:49.719413 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.719383 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/86854af6-6165-48b9-8ef0-e8b55a8e52aa-secret-metrics-server-client-certs\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.719526 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.719427 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/86854af6-6165-48b9-8ef0-e8b55a8e52aa-secret-metrics-server-tls\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.719526 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.719454 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86854af6-6165-48b9-8ef0-e8b55a8e52aa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.719652 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.719585 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/86854af6-6165-48b9-8ef0-e8b55a8e52aa-metrics-server-audit-profiles\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.719652 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.719620 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d987n\" (UniqueName: \"kubernetes.io/projected/86854af6-6165-48b9-8ef0-e8b55a8e52aa-kube-api-access-d987n\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.719742 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.719650 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/86854af6-6165-48b9-8ef0-e8b55a8e52aa-client-ca-bundle\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.719742 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.719704 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/86854af6-6165-48b9-8ef0-e8b55a8e52aa-audit-log\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.820999 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.820970 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/86854af6-6165-48b9-8ef0-e8b55a8e52aa-audit-log\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.821163 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.821030 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/86854af6-6165-48b9-8ef0-e8b55a8e52aa-secret-metrics-server-client-certs\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.821163 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.821057 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/86854af6-6165-48b9-8ef0-e8b55a8e52aa-secret-metrics-server-tls\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.821163 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.821079 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86854af6-6165-48b9-8ef0-e8b55a8e52aa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.821163 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.821128 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/86854af6-6165-48b9-8ef0-e8b55a8e52aa-metrics-server-audit-profiles\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.821433 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.821410 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/86854af6-6165-48b9-8ef0-e8b55a8e52aa-audit-log\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.821492 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.821461 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d987n\" (UniqueName: \"kubernetes.io/projected/86854af6-6165-48b9-8ef0-e8b55a8e52aa-kube-api-access-d987n\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.821537 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.821488 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/86854af6-6165-48b9-8ef0-e8b55a8e52aa-client-ca-bundle\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.822341 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.822313 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86854af6-6165-48b9-8ef0-e8b55a8e52aa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.822476 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.822447 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/86854af6-6165-48b9-8ef0-e8b55a8e52aa-metrics-server-audit-profiles\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.824177 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.824156 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/86854af6-6165-48b9-8ef0-e8b55a8e52aa-client-ca-bundle\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.824439 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.824419 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/86854af6-6165-48b9-8ef0-e8b55a8e52aa-secret-metrics-server-tls\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.824521 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.824485 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/86854af6-6165-48b9-8ef0-e8b55a8e52aa-secret-metrics-server-client-certs\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.830219 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.830195 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d987n\" (UniqueName: \"kubernetes.io/projected/86854af6-6165-48b9-8ef0-e8b55a8e52aa-kube-api-access-d987n\") pod \"metrics-server-79fd676bf4-5kjr6\" (UID: \"86854af6-6165-48b9-8ef0-e8b55a8e52aa\") " pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.956981 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.956896 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:46:49.975929 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.975899 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xpxdv"] Jul 20 16:46:49.978869 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.978845 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xpxdv" Jul 20 16:46:49.983350 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.983083 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-98x2w\"" Jul 20 16:46:49.987309 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.987281 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 20 16:46:49.991390 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:49.991360 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xpxdv"] Jul 20 16:46:50.023430 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.023401 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnrjd\" (UniqueName: \"kubernetes.io/projected/595b12c7-aa06-4bf8-96dd-34c503efa515-kube-api-access-gnrjd\") pod \"monitoring-plugin-7b7b7bc8bf-xpxdv\" (UID: \"595b12c7-aa06-4bf8-96dd-34c503efa515\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xpxdv" Jul 20 16:46:50.023562 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.023495 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/595b12c7-aa06-4bf8-96dd-34c503efa515-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-xpxdv\" (UID: \"595b12c7-aa06-4bf8-96dd-34c503efa515\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xpxdv" Jul 20 16:46:50.118564 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.118533 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79fd676bf4-5kjr6"] Jul 20 16:46:50.120215 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:46:50.120185 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86854af6_6165_48b9_8ef0_e8b55a8e52aa.slice/crio-83ad4a0e1ea3752ec8e31bb011cc880259d003aa7a88d194e4ffcb5aca634635 WatchSource:0}: Error finding container 83ad4a0e1ea3752ec8e31bb011cc880259d003aa7a88d194e4ffcb5aca634635: Status 404 returned error can't find the container with id 83ad4a0e1ea3752ec8e31bb011cc880259d003aa7a88d194e4ffcb5aca634635 Jul 20 16:46:50.124164 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.124137 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnrjd\" (UniqueName: \"kubernetes.io/projected/595b12c7-aa06-4bf8-96dd-34c503efa515-kube-api-access-gnrjd\") pod \"monitoring-plugin-7b7b7bc8bf-xpxdv\" (UID: \"595b12c7-aa06-4bf8-96dd-34c503efa515\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xpxdv" Jul 20 16:46:50.124266 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.124252 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/595b12c7-aa06-4bf8-96dd-34c503efa515-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-xpxdv\" (UID: \"595b12c7-aa06-4bf8-96dd-34c503efa515\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xpxdv" Jul 20 16:46:50.126724 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.126704 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/595b12c7-aa06-4bf8-96dd-34c503efa515-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-xpxdv\" (UID: \"595b12c7-aa06-4bf8-96dd-34c503efa515\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xpxdv" Jul 20 16:46:50.131145 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.131127 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnrjd\" (UniqueName: \"kubernetes.io/projected/595b12c7-aa06-4bf8-96dd-34c503efa515-kube-api-access-gnrjd\") pod \"monitoring-plugin-7b7b7bc8bf-xpxdv\" (UID: \"595b12c7-aa06-4bf8-96dd-34c503efa515\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xpxdv" Jul 20 16:46:50.166278 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.166244 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" event={"ID":"86854af6-6165-48b9-8ef0-e8b55a8e52aa","Type":"ContainerStarted","Data":"83ad4a0e1ea3752ec8e31bb011cc880259d003aa7a88d194e4ffcb5aca634635"} Jul 20 16:46:50.167806 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.167761 2566 generic.go:356] "Generic (PLEG): container finished" podID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerID="9bb14fe6c03605c75bcafd042d14d38d4064d611efd7f51212a52e428025fd03" exitCode=0 Jul 20 16:46:50.167879 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.167851 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c49477a4-db75-4ac8-80fe-159ce55be73c","Type":"ContainerDied","Data":"9bb14fe6c03605c75bcafd042d14d38d4064d611efd7f51212a52e428025fd03"} Jul 20 16:46:50.170200 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.170171 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" event={"ID":"6c716602-e8f7-4a96-ba67-b64a6126a0a2","Type":"ContainerStarted","Data":"39b808cb070adfe43f2177eeb1993f7e2db5e932918c0aa1cca359d40f15761c"} Jul 20 16:46:50.210012 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.209908 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gkfd9" podStartSLOduration=2.336804875 podStartE2EDuration="5.209861411s" totalImagesPullingTime="2.873056536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:50.208052227 +0000 UTC m=+74.994051305" watchObservedRunningTime="2026-07-20 16:46:50.209861411 +0000 UTC m=+74.995860490" Jul 20 16:46:50.304558 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.304525 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xpxdv" Jul 20 16:46:50.440438 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.440405 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xpxdv"] Jul 20 16:46:50.443180 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:46:50.443145 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod595b12c7_aa06_4bf8_96dd_34c503efa515.slice/crio-97a34bb4aaaebabd7f8502e227e4f67ba3e30ba6ad61898cb70f9221dca420bc WatchSource:0}: Error finding container 97a34bb4aaaebabd7f8502e227e4f67ba3e30ba6ad61898cb70f9221dca420bc: Status 404 returned error can't find the container with id 97a34bb4aaaebabd7f8502e227e4f67ba3e30ba6ad61898cb70f9221dca420bc Jul 20 16:46:50.456998 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.456971 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-86f5554fbd-88fdq"] Jul 20 16:46:50.460378 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.460327 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.463892 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.463857 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 20 16:46:50.463892 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.463883 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-vv96r\"" Jul 20 16:46:50.464057 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.463941 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 20 16:46:50.464057 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.463884 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 20 16:46:50.464057 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.463986 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 20 16:46:50.464213 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.464200 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 20 16:46:50.469818 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.469796 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 20 16:46:50.476588 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.476562 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-86f5554fbd-88fdq"] Jul 20 16:46:50.529733 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.529702 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8196778f-460c-4050-80d5-950ec200e832-serving-certs-ca-bundle\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.529890 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.529748 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8196778f-460c-4050-80d5-950ec200e832-federate-client-tls\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.529890 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.529788 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8196778f-460c-4050-80d5-950ec200e832-metrics-client-ca\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.529890 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.529860 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8196778f-460c-4050-80d5-950ec200e832-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.530023 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.529935 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlhn2\" (UniqueName: \"kubernetes.io/projected/8196778f-460c-4050-80d5-950ec200e832-kube-api-access-qlhn2\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.530023 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.529965 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8196778f-460c-4050-80d5-950ec200e832-telemeter-trusted-ca-bundle\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.530023 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.529998 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8196778f-460c-4050-80d5-950ec200e832-secret-telemeter-client\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.530128 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.530026 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8196778f-460c-4050-80d5-950ec200e832-telemeter-client-tls\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.631447 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.631413 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8196778f-460c-4050-80d5-950ec200e832-telemeter-client-tls\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.631642 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.631476 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8196778f-460c-4050-80d5-950ec200e832-serving-certs-ca-bundle\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.631642 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.631528 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8196778f-460c-4050-80d5-950ec200e832-federate-client-tls\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.631642 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.631551 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8196778f-460c-4050-80d5-950ec200e832-metrics-client-ca\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.631642 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.631579 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8196778f-460c-4050-80d5-950ec200e832-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.631642 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.631618 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlhn2\" (UniqueName: \"kubernetes.io/projected/8196778f-460c-4050-80d5-950ec200e832-kube-api-access-qlhn2\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.631932 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.631644 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8196778f-460c-4050-80d5-950ec200e832-telemeter-trusted-ca-bundle\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.631932 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.631680 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8196778f-460c-4050-80d5-950ec200e832-secret-telemeter-client\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.632753 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.632696 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8196778f-460c-4050-80d5-950ec200e832-metrics-client-ca\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.632753 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.632726 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8196778f-460c-4050-80d5-950ec200e832-serving-certs-ca-bundle\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.633060 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.633021 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8196778f-460c-4050-80d5-950ec200e832-telemeter-trusted-ca-bundle\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.634735 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.634713 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8196778f-460c-4050-80d5-950ec200e832-telemeter-client-tls\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.634855 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.634765 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8196778f-460c-4050-80d5-950ec200e832-federate-client-tls\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.635118 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.635099 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8196778f-460c-4050-80d5-950ec200e832-secret-telemeter-client\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.635434 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.635413 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8196778f-460c-4050-80d5-950ec200e832-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.647909 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.647886 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlhn2\" (UniqueName: \"kubernetes.io/projected/8196778f-460c-4050-80d5-950ec200e832-kube-api-access-qlhn2\") pod \"telemeter-client-86f5554fbd-88fdq\" (UID: \"8196778f-460c-4050-80d5-950ec200e832\") " pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:50.771764 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:50.771689 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" Jul 20 16:46:51.182854 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:51.182599 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xpxdv" event={"ID":"595b12c7-aa06-4bf8-96dd-34c503efa515","Type":"ContainerStarted","Data":"97a34bb4aaaebabd7f8502e227e4f67ba3e30ba6ad61898cb70f9221dca420bc"} Jul 20 16:46:51.245621 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:51.245592 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-86f5554fbd-88fdq"] Jul 20 16:46:51.251704 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:46:51.251675 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8196778f_460c_4050_80d5_950ec200e832.slice/crio-0de55beb65c0bb142925d293380e1600b65812feb396b7629cf28e191f0f68b8 WatchSource:0}: Error finding container 0de55beb65c0bb142925d293380e1600b65812feb396b7629cf28e191f0f68b8: Status 404 returned error can't find the container with id 0de55beb65c0bb142925d293380e1600b65812feb396b7629cf28e191f0f68b8 Jul 20 16:46:52.190825 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:52.190688 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" event={"ID":"86854af6-6165-48b9-8ef0-e8b55a8e52aa","Type":"ContainerStarted","Data":"5465cf7c14b0cbc620fe5167760dc080ef8a162e9e8b010fe466f9dbe78360cd"} Jul 20 16:46:52.196228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:52.196195 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" event={"ID":"8d7b1577-accd-455c-8511-9198592e5c51","Type":"ContainerStarted","Data":"4161496425d7e6d2e844ae9fef5a5113d69a0c18bd1759b442037fbdf2902296"} Jul 20 16:46:52.196373 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:52.196275 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" event={"ID":"8d7b1577-accd-455c-8511-9198592e5c51","Type":"ContainerStarted","Data":"2cef47c6e15edcad41eddf7ba39ef50624bedf4cb8b0bed489ed41012cceabf8"} Jul 20 16:46:52.196373 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:52.196348 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" event={"ID":"8d7b1577-accd-455c-8511-9198592e5c51","Type":"ContainerStarted","Data":"972f51ceca2ce9a0a2e3a70e9422121633f4b9fe3f159cd894ee3cfefecdf02c"} Jul 20 16:46:52.198441 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:52.198413 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" event={"ID":"8196778f-460c-4050-80d5-950ec200e832","Type":"ContainerStarted","Data":"0de55beb65c0bb142925d293380e1600b65812feb396b7629cf28e191f0f68b8"} Jul 20 16:46:52.208712 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:52.208644 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" podStartSLOduration=1.421881034 podStartE2EDuration="3.208625937s" totalImagesPullingTime="1.786744903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:52.207803676 +0000 UTC m=+76.993802753" watchObservedRunningTime="2026-07-20 16:46:52.208625937 +0000 UTC m=+76.994625017" Jul 20 16:46:53.207303 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:53.207215 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c49477a4-db75-4ac8-80fe-159ce55be73c","Type":"ContainerStarted","Data":"cc92ad4388386f1cb521161f43d8b97ffb7b545c96b910bf25b92621d665a13c"} Jul 20 16:46:53.207303 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:53.207298 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c49477a4-db75-4ac8-80fe-159ce55be73c","Type":"ContainerStarted","Data":"7829bdcfa48f9bd3aa0cb7d75dddc147ca0d3c8bd392843c289ca1127d75c2b9"} Jul 20 16:46:53.207847 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:53.207340 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c49477a4-db75-4ac8-80fe-159ce55be73c","Type":"ContainerStarted","Data":"64f3a05495e83161618ac79728c9b748384c8a090d9108d0dd5abcea3d06d686"} Jul 20 16:46:53.207847 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:53.207375 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c49477a4-db75-4ac8-80fe-159ce55be73c","Type":"ContainerStarted","Data":"590cafd2a18f14da1ee2ec104d701e2a728ca6f8b8170f666746c6b539f09f93"} Jul 20 16:46:53.207847 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:53.207407 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c49477a4-db75-4ac8-80fe-159ce55be73c","Type":"ContainerStarted","Data":"49dba45c7b7bc1ac91ee4879816558f398e80ee0dd9a0093eef86e4a7b5f4920"} Jul 20 16:46:53.207847 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:53.207442 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c49477a4-db75-4ac8-80fe-159ce55be73c","Type":"ContainerStarted","Data":"3ef446f24eb8277ea49a49440fea09bcfb45368efe1c427353fef9e49a04f0a1"} Jul 20 16:46:53.211059 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:53.210845 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" event={"ID":"8d7b1577-accd-455c-8511-9198592e5c51","Type":"ContainerStarted","Data":"d77985399b787bb12eaa9ca647c5206dffa316869b988b7434260959577fcb5e"} Jul 20 16:46:53.211059 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:53.210921 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" event={"ID":"8d7b1577-accd-455c-8511-9198592e5c51","Type":"ContainerStarted","Data":"088daadbeb51d54fb4ee9b197fd574802fbbd5167f378485ad4379521ec278a5"} Jul 20 16:46:53.211059 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:53.210965 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" event={"ID":"8d7b1577-accd-455c-8511-9198592e5c51","Type":"ContainerStarted","Data":"5467de7cd54a5652b24ae28a0875b9a8a671e08c656cdcabfca09d4d27e97c00"} Jul 20 16:46:53.211059 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:53.211022 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:46:53.237856 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:53.237798 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.8744250949999999 podStartE2EDuration="7.237755275s" totalImagesPullingTime="5.341717672s" totalInitContainerRuntime="21.612508ms" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:53.23397412 +0000 UTC m=+78.019973199" watchObservedRunningTime="2026-07-20 16:46:53.237755275 +0000 UTC m=+78.023754355" Jul 20 16:46:54.023926 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.023869 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" podStartSLOduration=2.377437233 podStartE2EDuration="6.023849751s" totalImagesPullingTime="3.646412518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:53.256651072 +0000 UTC m=+78.042650148" watchObservedRunningTime="2026-07-20 16:46:54.023849751 +0000 UTC m=+78.809848839" Jul 20 16:46:54.024128 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.024090 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f5d69d4fd-mbdvm"] Jul 20 16:46:54.055570 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.055529 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-775c7445f7-svst4"] Jul 20 16:46:54.059155 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.059129 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.067579 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.067547 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-775c7445f7-svst4"] Jul 20 16:46:54.175538 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.175513 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00742c8a-f4c0-4804-a13e-254428723a76-console-serving-cert\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.175672 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.175564 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00742c8a-f4c0-4804-a13e-254428723a76-console-oauth-config\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.175757 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.175728 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bsfl\" (UniqueName: \"kubernetes.io/projected/00742c8a-f4c0-4804-a13e-254428723a76-kube-api-access-9bsfl\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.175882 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.175802 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-service-ca\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.175882 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.175834 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-trusted-ca-bundle\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.175980 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.175921 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-console-config\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.176027 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.175971 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-oauth-serving-cert\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.217666 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.217641 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xpxdv" event={"ID":"595b12c7-aa06-4bf8-96dd-34c503efa515","Type":"ContainerStarted","Data":"e0817d4096cf36a7bda469b26fe42c3505f3496e7b5cf1e86a69af89312145ab"} Jul 20 16:46:54.219254 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.219224 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" event={"ID":"8196778f-460c-4050-80d5-950ec200e832","Type":"ContainerStarted","Data":"71c1e1ec36f89413ad4953a0268b219c0132c68d643826acba8d742377e7efd9"} Jul 20 16:46:54.220923 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.220902 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xpxdv" Jul 20 16:46:54.221971 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.221947 2566 patch_prober.go:28] interesting pod/monitoring-plugin-7b7b7bc8bf-xpxdv container/monitoring-plugin namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.134.0.18:9443/health\": dial tcp 10.134.0.18:9443: connect: connection refused" start-of-body= Jul 20 16:46:54.222064 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.221991 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xpxdv" podUID="595b12c7-aa06-4bf8-96dd-34c503efa515" containerName="monitoring-plugin" probeResult="failure" output="Get \"https://10.134.0.18:9443/health\": dial tcp 10.134.0.18:9443: connect: connection refused" Jul 20 16:46:54.238391 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.238342 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xpxdv" podStartSLOduration=1.5479039430000001 podStartE2EDuration="5.238322763s" totalImagesPullingTime="3.69041882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:54.237902322 +0000 UTC m=+79.023901406" watchObservedRunningTime="2026-07-20 16:46:54.238322763 +0000 UTC m=+79.024321843" Jul 20 16:46:54.276820 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.276753 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bsfl\" (UniqueName: \"kubernetes.io/projected/00742c8a-f4c0-4804-a13e-254428723a76-kube-api-access-9bsfl\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.276820 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.276814 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-service-ca\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.277157 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.277129 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-trusted-ca-bundle\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.278485 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.277306 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-console-config\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.278485 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.277500 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-service-ca\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.278485 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.277598 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-oauth-serving-cert\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.278485 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.277696 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00742c8a-f4c0-4804-a13e-254428723a76-console-serving-cert\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.278485 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.277850 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00742c8a-f4c0-4804-a13e-254428723a76-console-oauth-config\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.278485 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.278028 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-console-config\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.278485 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.278197 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-trusted-ca-bundle\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.278485 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.278240 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-oauth-serving-cert\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.280504 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.280486 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00742c8a-f4c0-4804-a13e-254428723a76-console-oauth-config\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.280842 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.280825 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00742c8a-f4c0-4804-a13e-254428723a76-console-serving-cert\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.284037 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.284019 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bsfl\" (UniqueName: \"kubernetes.io/projected/00742c8a-f4c0-4804-a13e-254428723a76-kube-api-access-9bsfl\") pod \"console-775c7445f7-svst4\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.371990 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.371946 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:46:54.500794 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:54.500715 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-775c7445f7-svst4"] Jul 20 16:46:54.502848 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:46:54.502819 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00742c8a_f4c0_4804_a13e_254428723a76.slice/crio-bab76a64c8f203f27e286b490cad97a3e575f539475e92692b6de852381a8614 WatchSource:0}: Error finding container bab76a64c8f203f27e286b490cad97a3e575f539475e92692b6de852381a8614: Status 404 returned error can't find the container with id bab76a64c8f203f27e286b490cad97a3e575f539475e92692b6de852381a8614 Jul 20 16:46:55.230044 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:55.230005 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" event={"ID":"8196778f-460c-4050-80d5-950ec200e832","Type":"ContainerStarted","Data":"ad389aecfcd8fc9b107bd9857b9427585cf0e2629b75e8d3c2c0cc9a13e33e94"} Jul 20 16:46:55.230540 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:55.230103 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" event={"ID":"8196778f-460c-4050-80d5-950ec200e832","Type":"ContainerStarted","Data":"82dd988a1ebea9ca25c92a6fa847c81775129d5bb6b3e2f0cb88af1302f9f3d5"} Jul 20 16:46:55.231637 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:55.231608 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-775c7445f7-svst4" event={"ID":"00742c8a-f4c0-4804-a13e-254428723a76","Type":"ContainerStarted","Data":"34913b40a02bf9ef287ad0f5ac81c8b832285d0cefea40aa4ba327d87bfc9040"} Jul 20 16:46:55.231754 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:55.231655 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-775c7445f7-svst4" event={"ID":"00742c8a-f4c0-4804-a13e-254428723a76","Type":"ContainerStarted","Data":"bab76a64c8f203f27e286b490cad97a3e575f539475e92692b6de852381a8614"} Jul 20 16:46:55.236120 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:55.236091 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xpxdv" Jul 20 16:46:55.250362 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:55.250314 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-86f5554fbd-88fdq" podStartSLOduration=2.364464811 podStartE2EDuration="5.250301003s" totalImagesPullingTime="2.885836192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:55.250013431 +0000 UTC m=+80.036012511" watchObservedRunningTime="2026-07-20 16:46:55.250301003 +0000 UTC m=+80.036300081" Jul 20 16:46:55.269112 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:55.269063 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-775c7445f7-svst4" podStartSLOduration=1.2690489409999999 podStartE2EDuration="1.269048941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:55.267465056 +0000 UTC m=+80.053464138" watchObservedRunningTime="2026-07-20 16:46:55.269048941 +0000 UTC m=+80.055048126" Jul 20 16:46:55.549260 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:55.549186 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:46:59.226273 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:46:59.226159 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-fbb45cdcd-dz559" Jul 20 16:47:01.271109 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.271070 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-775c7445f7-svst4"] Jul 20 16:47:01.298250 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.298215 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68ffc7f898-j4bwj"] Jul 20 16:47:01.316856 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.316822 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68ffc7f898-j4bwj"] Jul 20 16:47:01.317002 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.316917 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.439920 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.439881 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-console-oauth-config\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.440080 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.439940 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-trusted-ca-bundle\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.440080 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.440008 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-console-config\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.440080 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.440032 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-service-ca\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.440080 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.440049 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-oauth-serving-cert\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.440080 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.440067 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x59dt\" (UniqueName: \"kubernetes.io/projected/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-kube-api-access-x59dt\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.440276 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.440086 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-console-serving-cert\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.540744 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.540652 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-console-serving-cert\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.540744 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.540702 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-console-oauth-config\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.540744 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.540740 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-trusted-ca-bundle\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.541017 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.540799 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-console-config\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.541017 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.540832 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-service-ca\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.541017 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.540860 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-oauth-serving-cert\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.541017 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.540891 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x59dt\" (UniqueName: \"kubernetes.io/projected/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-kube-api-access-x59dt\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.541662 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.541630 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-console-config\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.541822 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.541690 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-service-ca\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.541889 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.541865 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-trusted-ca-bundle\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.541943 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.541888 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-oauth-serving-cert\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.543424 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.543395 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-console-oauth-config\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.543529 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.543514 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-console-serving-cert\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.548402 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.548383 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x59dt\" (UniqueName: \"kubernetes.io/projected/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-kube-api-access-x59dt\") pod \"console-68ffc7f898-j4bwj\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.626786 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.626747 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:01.797327 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:01.797135 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68ffc7f898-j4bwj"] Jul 20 16:47:01.799957 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:47:01.799928 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod308ef8a5_3aaf_42c5_b2b3_56a044947a8e.slice/crio-48f9ae395c8167c078341dba2d717d5a2450437982cb15feacd3abd32a126ca1 WatchSource:0}: Error finding container 48f9ae395c8167c078341dba2d717d5a2450437982cb15feacd3abd32a126ca1: Status 404 returned error can't find the container with id 48f9ae395c8167c078341dba2d717d5a2450437982cb15feacd3abd32a126ca1 Jul 20 16:47:02.263609 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:02.263570 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68ffc7f898-j4bwj" event={"ID":"308ef8a5-3aaf-42c5-b2b3-56a044947a8e","Type":"ContainerStarted","Data":"194c7510665dc941a989d821f329ab2fd428187c0d332bfb142ed22106429bc3"} Jul 20 16:47:02.263764 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:02.263625 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68ffc7f898-j4bwj" event={"ID":"308ef8a5-3aaf-42c5-b2b3-56a044947a8e","Type":"ContainerStarted","Data":"48f9ae395c8167c078341dba2d717d5a2450437982cb15feacd3abd32a126ca1"} Jul 20 16:47:02.288554 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:02.288504 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68ffc7f898-j4bwj" podStartSLOduration=1.288487197 podStartE2EDuration="1.288487197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:47:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:47:02.286116075 +0000 UTC m=+87.072115165" watchObservedRunningTime="2026-07-20 16:47:02.288487197 +0000 UTC m=+87.074486276" Jul 20 16:47:04.372893 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:04.372858 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:47:09.957572 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:09.957532 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:47:09.958059 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:09.957598 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:47:11.627593 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:11.627554 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:11.628022 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:11.627662 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:11.632587 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:11.632563 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:12.308500 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:12.308473 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:47:12.354230 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:12.354199 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-ddc59cf58-8smsw"] Jul 20 16:47:18.154743 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:18.154710 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qfqcm" Jul 20 16:47:19.047600 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.047556 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f5d69d4fd-mbdvm" podUID="f35f1455-d7b5-4e8f-99e6-841c4d3c30fd" containerName="console" containerID="cri-o://a137923e465026d030ad0f97cc127529988c4d806b432f55f7319e95279372a9" gracePeriod=15 Jul 20 16:47:19.813109 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.813077 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:47:19.898462 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.898423 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-console-oauth-config\") pod \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " Jul 20 16:47:19.898462 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.898463 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-console-serving-cert\") pod \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " Jul 20 16:47:19.898709 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.898509 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-trusted-ca-bundle\") pod \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " Jul 20 16:47:19.898709 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.898539 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-service-ca\") pod \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " Jul 20 16:47:19.898709 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.898554 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-oauth-serving-cert\") pod \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " Jul 20 16:47:19.898709 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.898580 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-console-config\") pod \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " Jul 20 16:47:19.898709 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.898601 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9w4x\" (UniqueName: \"kubernetes.io/projected/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-kube-api-access-r9w4x\") pod \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\" (UID: \"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd\") " Jul 20 16:47:19.899066 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.899029 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f35f1455-d7b5-4e8f-99e6-841c4d3c30fd" (UID: "f35f1455-d7b5-4e8f-99e6-841c4d3c30fd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:47:19.899132 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.899031 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-service-ca" (OuterVolumeSpecName: "service-ca") pod "f35f1455-d7b5-4e8f-99e6-841c4d3c30fd" (UID: "f35f1455-d7b5-4e8f-99e6-841c4d3c30fd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:47:19.899132 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.899074 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f35f1455-d7b5-4e8f-99e6-841c4d3c30fd" (UID: "f35f1455-d7b5-4e8f-99e6-841c4d3c30fd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:47:19.899132 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.899094 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-console-config" (OuterVolumeSpecName: "console-config") pod "f35f1455-d7b5-4e8f-99e6-841c4d3c30fd" (UID: "f35f1455-d7b5-4e8f-99e6-841c4d3c30fd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:47:19.901084 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.901056 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f35f1455-d7b5-4e8f-99e6-841c4d3c30fd" (UID: "f35f1455-d7b5-4e8f-99e6-841c4d3c30fd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:47:19.901176 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.901079 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-kube-api-access-r9w4x" (OuterVolumeSpecName: "kube-api-access-r9w4x") pod "f35f1455-d7b5-4e8f-99e6-841c4d3c30fd" (UID: "f35f1455-d7b5-4e8f-99e6-841c4d3c30fd"). InnerVolumeSpecName "kube-api-access-r9w4x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:47:19.901176 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.901106 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f35f1455-d7b5-4e8f-99e6-841c4d3c30fd" (UID: "f35f1455-d7b5-4e8f-99e6-841c4d3c30fd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:47:19.999303 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.999226 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-trusted-ca-bundle\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:19.999303 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.999257 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-service-ca\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:19.999303 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.999267 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-oauth-serving-cert\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:19.999303 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.999275 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-console-config\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:19.999303 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.999284 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9w4x\" (UniqueName: \"kubernetes.io/projected/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-kube-api-access-r9w4x\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:19.999303 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.999292 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-console-oauth-config\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:19.999303 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:19.999300 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd-console-serving-cert\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:20.341576 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:20.341540 2566 generic.go:356] "Generic (PLEG): container finished" podID="f35f1455-d7b5-4e8f-99e6-841c4d3c30fd" containerID="a137923e465026d030ad0f97cc127529988c4d806b432f55f7319e95279372a9" exitCode=0 Jul 20 16:47:20.341744 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:20.341583 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5d69d4fd-mbdvm" Jul 20 16:47:20.341744 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:20.341625 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5d69d4fd-mbdvm" event={"ID":"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd","Type":"ContainerDied","Data":"a137923e465026d030ad0f97cc127529988c4d806b432f55f7319e95279372a9"} Jul 20 16:47:20.341744 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:20.341675 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5d69d4fd-mbdvm" event={"ID":"f35f1455-d7b5-4e8f-99e6-841c4d3c30fd","Type":"ContainerDied","Data":"6ece1d46c56d5832589f7cf0eb362058908c2750ad2ba5099f3e784fa4962050"} Jul 20 16:47:20.341744 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:20.341695 2566 scope.go:117] "RemoveContainer" containerID="a137923e465026d030ad0f97cc127529988c4d806b432f55f7319e95279372a9" Jul 20 16:47:20.351070 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:20.351052 2566 scope.go:117] "RemoveContainer" containerID="a137923e465026d030ad0f97cc127529988c4d806b432f55f7319e95279372a9" Jul 20 16:47:20.351350 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:47:20.351325 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a137923e465026d030ad0f97cc127529988c4d806b432f55f7319e95279372a9\": container with ID starting with a137923e465026d030ad0f97cc127529988c4d806b432f55f7319e95279372a9 not found: ID does not exist" containerID="a137923e465026d030ad0f97cc127529988c4d806b432f55f7319e95279372a9" Jul 20 16:47:20.351396 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:20.351360 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a137923e465026d030ad0f97cc127529988c4d806b432f55f7319e95279372a9"} err="failed to get container status \"a137923e465026d030ad0f97cc127529988c4d806b432f55f7319e95279372a9\": rpc error: code = NotFound desc = could not find container \"a137923e465026d030ad0f97cc127529988c4d806b432f55f7319e95279372a9\": container with ID starting with a137923e465026d030ad0f97cc127529988c4d806b432f55f7319e95279372a9 not found: ID does not exist" Jul 20 16:47:20.361636 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:20.361607 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f5d69d4fd-mbdvm"] Jul 20 16:47:20.366103 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:20.366085 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f5d69d4fd-mbdvm"] Jul 20 16:47:21.813892 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:21.813861 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f35f1455-d7b5-4e8f-99e6-841c4d3c30fd" path="/var/lib/kubelet/pods/f35f1455-d7b5-4e8f-99e6-841c4d3c30fd/volumes" Jul 20 16:47:26.292009 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.291968 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-775c7445f7-svst4" podUID="00742c8a-f4c0-4804-a13e-254428723a76" containerName="console" containerID="cri-o://34913b40a02bf9ef287ad0f5ac81c8b832285d0cefea40aa4ba327d87bfc9040" gracePeriod=15 Jul 20 16:47:26.538823 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.538794 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:47:26.655016 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.654978 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-service-ca\") pod \"00742c8a-f4c0-4804-a13e-254428723a76\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " Jul 20 16:47:26.655016 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.655014 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-oauth-serving-cert\") pod \"00742c8a-f4c0-4804-a13e-254428723a76\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " Jul 20 16:47:26.655264 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.655044 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-trusted-ca-bundle\") pod \"00742c8a-f4c0-4804-a13e-254428723a76\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " Jul 20 16:47:26.655264 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.655098 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-console-config\") pod \"00742c8a-f4c0-4804-a13e-254428723a76\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " Jul 20 16:47:26.655264 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.655121 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bsfl\" (UniqueName: \"kubernetes.io/projected/00742c8a-f4c0-4804-a13e-254428723a76-kube-api-access-9bsfl\") pod \"00742c8a-f4c0-4804-a13e-254428723a76\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " Jul 20 16:47:26.655264 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.655154 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00742c8a-f4c0-4804-a13e-254428723a76-console-serving-cert\") pod \"00742c8a-f4c0-4804-a13e-254428723a76\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " Jul 20 16:47:26.655464 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.655316 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00742c8a-f4c0-4804-a13e-254428723a76-console-oauth-config\") pod \"00742c8a-f4c0-4804-a13e-254428723a76\" (UID: \"00742c8a-f4c0-4804-a13e-254428723a76\") " Jul 20 16:47:26.655464 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.655413 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "00742c8a-f4c0-4804-a13e-254428723a76" (UID: "00742c8a-f4c0-4804-a13e-254428723a76"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:47:26.655563 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.655458 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-service-ca" (OuterVolumeSpecName: "service-ca") pod "00742c8a-f4c0-4804-a13e-254428723a76" (UID: "00742c8a-f4c0-4804-a13e-254428723a76"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:47:26.655563 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.655531 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-console-config" (OuterVolumeSpecName: "console-config") pod "00742c8a-f4c0-4804-a13e-254428723a76" (UID: "00742c8a-f4c0-4804-a13e-254428723a76"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:47:26.655635 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.655572 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "00742c8a-f4c0-4804-a13e-254428723a76" (UID: "00742c8a-f4c0-4804-a13e-254428723a76"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:47:26.655635 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.655627 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-console-config\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:26.655741 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.655640 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-service-ca\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:26.655741 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.655648 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-oauth-serving-cert\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:26.657414 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.657392 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00742c8a-f4c0-4804-a13e-254428723a76-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "00742c8a-f4c0-4804-a13e-254428723a76" (UID: "00742c8a-f4c0-4804-a13e-254428723a76"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:47:26.657500 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.657450 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00742c8a-f4c0-4804-a13e-254428723a76-kube-api-access-9bsfl" (OuterVolumeSpecName: "kube-api-access-9bsfl") pod "00742c8a-f4c0-4804-a13e-254428723a76" (UID: "00742c8a-f4c0-4804-a13e-254428723a76"). InnerVolumeSpecName "kube-api-access-9bsfl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:47:26.657539 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.657496 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00742c8a-f4c0-4804-a13e-254428723a76-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "00742c8a-f4c0-4804-a13e-254428723a76" (UID: "00742c8a-f4c0-4804-a13e-254428723a76"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:47:26.756317 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.756269 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00742c8a-f4c0-4804-a13e-254428723a76-console-oauth-config\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:26.756317 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.756312 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00742c8a-f4c0-4804-a13e-254428723a76-trusted-ca-bundle\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:26.756317 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.756322 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bsfl\" (UniqueName: \"kubernetes.io/projected/00742c8a-f4c0-4804-a13e-254428723a76-kube-api-access-9bsfl\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:26.756317 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:26.756332 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00742c8a-f4c0-4804-a13e-254428723a76-console-serving-cert\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:27.372150 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:27.372114 2566 generic.go:356] "Generic (PLEG): container finished" podID="00742c8a-f4c0-4804-a13e-254428723a76" containerID="34913b40a02bf9ef287ad0f5ac81c8b832285d0cefea40aa4ba327d87bfc9040" exitCode=0 Jul 20 16:47:27.372626 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:27.372168 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-775c7445f7-svst4" Jul 20 16:47:27.372626 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:27.372166 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-775c7445f7-svst4" event={"ID":"00742c8a-f4c0-4804-a13e-254428723a76","Type":"ContainerDied","Data":"34913b40a02bf9ef287ad0f5ac81c8b832285d0cefea40aa4ba327d87bfc9040"} Jul 20 16:47:27.372626 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:27.372228 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-775c7445f7-svst4" event={"ID":"00742c8a-f4c0-4804-a13e-254428723a76","Type":"ContainerDied","Data":"bab76a64c8f203f27e286b490cad97a3e575f539475e92692b6de852381a8614"} Jul 20 16:47:27.372626 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:27.372254 2566 scope.go:117] "RemoveContainer" containerID="34913b40a02bf9ef287ad0f5ac81c8b832285d0cefea40aa4ba327d87bfc9040" Jul 20 16:47:27.381068 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:27.381052 2566 scope.go:117] "RemoveContainer" containerID="34913b40a02bf9ef287ad0f5ac81c8b832285d0cefea40aa4ba327d87bfc9040" Jul 20 16:47:27.381360 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:47:27.381343 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34913b40a02bf9ef287ad0f5ac81c8b832285d0cefea40aa4ba327d87bfc9040\": container with ID starting with 34913b40a02bf9ef287ad0f5ac81c8b832285d0cefea40aa4ba327d87bfc9040 not found: ID does not exist" containerID="34913b40a02bf9ef287ad0f5ac81c8b832285d0cefea40aa4ba327d87bfc9040" Jul 20 16:47:27.381406 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:27.381370 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34913b40a02bf9ef287ad0f5ac81c8b832285d0cefea40aa4ba327d87bfc9040"} err="failed to get container status \"34913b40a02bf9ef287ad0f5ac81c8b832285d0cefea40aa4ba327d87bfc9040\": rpc error: code = NotFound desc = could not find container \"34913b40a02bf9ef287ad0f5ac81c8b832285d0cefea40aa4ba327d87bfc9040\": container with ID starting with 34913b40a02bf9ef287ad0f5ac81c8b832285d0cefea40aa4ba327d87bfc9040 not found: ID does not exist" Jul 20 16:47:27.392419 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:27.392395 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-775c7445f7-svst4"] Jul 20 16:47:27.398103 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:27.398079 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-775c7445f7-svst4"] Jul 20 16:47:27.814831 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:27.814736 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00742c8a-f4c0-4804-a13e-254428723a76" path="/var/lib/kubelet/pods/00742c8a-f4c0-4804-a13e-254428723a76/volumes" Jul 20 16:47:29.963766 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:29.963736 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:47:29.967867 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:29.967839 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-79fd676bf4-5kjr6" Jul 20 16:47:37.378991 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.378946 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-ddc59cf58-8smsw" podUID="896660eb-da4e-4fb5-b0df-7bfa50ed6477" containerName="console" containerID="cri-o://81ebd5140b3085cf76f536171b223ade9193ff521322aa53a6493fac1ba3408d" gracePeriod=15 Jul 20 16:47:37.627948 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.627923 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:47:37.754691 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.754590 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/896660eb-da4e-4fb5-b0df-7bfa50ed6477-service-ca\") pod \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " Jul 20 16:47:37.754691 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.754658 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8rqx\" (UniqueName: \"kubernetes.io/projected/896660eb-da4e-4fb5-b0df-7bfa50ed6477-kube-api-access-s8rqx\") pod \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " Jul 20 16:47:37.754691 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.754698 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/896660eb-da4e-4fb5-b0df-7bfa50ed6477-console-config\") pod \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " Jul 20 16:47:37.754980 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.754729 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/896660eb-da4e-4fb5-b0df-7bfa50ed6477-console-serving-cert\") pod \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " Jul 20 16:47:37.754980 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.754754 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/896660eb-da4e-4fb5-b0df-7bfa50ed6477-console-oauth-config\") pod \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " Jul 20 16:47:37.754980 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.754820 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/896660eb-da4e-4fb5-b0df-7bfa50ed6477-oauth-serving-cert\") pod \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\" (UID: \"896660eb-da4e-4fb5-b0df-7bfa50ed6477\") " Jul 20 16:47:37.755234 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.755104 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/896660eb-da4e-4fb5-b0df-7bfa50ed6477-service-ca" (OuterVolumeSpecName: "service-ca") pod "896660eb-da4e-4fb5-b0df-7bfa50ed6477" (UID: "896660eb-da4e-4fb5-b0df-7bfa50ed6477"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:47:37.755313 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.755242 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/896660eb-da4e-4fb5-b0df-7bfa50ed6477-console-config" (OuterVolumeSpecName: "console-config") pod "896660eb-da4e-4fb5-b0df-7bfa50ed6477" (UID: "896660eb-da4e-4fb5-b0df-7bfa50ed6477"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:47:37.755313 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.755283 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/896660eb-da4e-4fb5-b0df-7bfa50ed6477-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "896660eb-da4e-4fb5-b0df-7bfa50ed6477" (UID: "896660eb-da4e-4fb5-b0df-7bfa50ed6477"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:47:37.757088 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.757065 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/896660eb-da4e-4fb5-b0df-7bfa50ed6477-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "896660eb-da4e-4fb5-b0df-7bfa50ed6477" (UID: "896660eb-da4e-4fb5-b0df-7bfa50ed6477"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:47:37.757174 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.757146 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/896660eb-da4e-4fb5-b0df-7bfa50ed6477-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "896660eb-da4e-4fb5-b0df-7bfa50ed6477" (UID: "896660eb-da4e-4fb5-b0df-7bfa50ed6477"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:47:37.757174 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.757164 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/896660eb-da4e-4fb5-b0df-7bfa50ed6477-kube-api-access-s8rqx" (OuterVolumeSpecName: "kube-api-access-s8rqx") pod "896660eb-da4e-4fb5-b0df-7bfa50ed6477" (UID: "896660eb-da4e-4fb5-b0df-7bfa50ed6477"). InnerVolumeSpecName "kube-api-access-s8rqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:47:37.856238 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.856207 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8rqx\" (UniqueName: \"kubernetes.io/projected/896660eb-da4e-4fb5-b0df-7bfa50ed6477-kube-api-access-s8rqx\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:37.856238 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.856234 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/896660eb-da4e-4fb5-b0df-7bfa50ed6477-console-config\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:37.856238 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.856243 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/896660eb-da4e-4fb5-b0df-7bfa50ed6477-console-serving-cert\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:37.856437 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.856253 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/896660eb-da4e-4fb5-b0df-7bfa50ed6477-console-oauth-config\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:37.856437 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.856261 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/896660eb-da4e-4fb5-b0df-7bfa50ed6477-oauth-serving-cert\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:37.856437 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:37.856269 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/896660eb-da4e-4fb5-b0df-7bfa50ed6477-service-ca\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:47:38.425379 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:38.425343 2566 generic.go:356] "Generic (PLEG): container finished" podID="896660eb-da4e-4fb5-b0df-7bfa50ed6477" containerID="81ebd5140b3085cf76f536171b223ade9193ff521322aa53a6493fac1ba3408d" exitCode=0 Jul 20 16:47:38.425803 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:38.425379 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ddc59cf58-8smsw" event={"ID":"896660eb-da4e-4fb5-b0df-7bfa50ed6477","Type":"ContainerDied","Data":"81ebd5140b3085cf76f536171b223ade9193ff521322aa53a6493fac1ba3408d"} Jul 20 16:47:38.425803 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:38.425434 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ddc59cf58-8smsw" event={"ID":"896660eb-da4e-4fb5-b0df-7bfa50ed6477","Type":"ContainerDied","Data":"bf3b237796573ade86ca49750b59f939f5f47622986d085f212396cd04bd0cfa"} Jul 20 16:47:38.425803 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:38.425455 2566 scope.go:117] "RemoveContainer" containerID="81ebd5140b3085cf76f536171b223ade9193ff521322aa53a6493fac1ba3408d" Jul 20 16:47:38.425803 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:38.425391 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ddc59cf58-8smsw" Jul 20 16:47:38.433869 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:38.433851 2566 scope.go:117] "RemoveContainer" containerID="81ebd5140b3085cf76f536171b223ade9193ff521322aa53a6493fac1ba3408d" Jul 20 16:47:38.434124 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:47:38.434105 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81ebd5140b3085cf76f536171b223ade9193ff521322aa53a6493fac1ba3408d\": container with ID starting with 81ebd5140b3085cf76f536171b223ade9193ff521322aa53a6493fac1ba3408d not found: ID does not exist" containerID="81ebd5140b3085cf76f536171b223ade9193ff521322aa53a6493fac1ba3408d" Jul 20 16:47:38.434196 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:38.434139 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81ebd5140b3085cf76f536171b223ade9193ff521322aa53a6493fac1ba3408d"} err="failed to get container status \"81ebd5140b3085cf76f536171b223ade9193ff521322aa53a6493fac1ba3408d\": rpc error: code = NotFound desc = could not find container \"81ebd5140b3085cf76f536171b223ade9193ff521322aa53a6493fac1ba3408d\": container with ID starting with 81ebd5140b3085cf76f536171b223ade9193ff521322aa53a6493fac1ba3408d not found: ID does not exist" Jul 20 16:47:38.440475 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:38.440452 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-ddc59cf58-8smsw"] Jul 20 16:47:38.443708 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:38.443687 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-ddc59cf58-8smsw"] Jul 20 16:47:39.814162 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:47:39.814117 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="896660eb-da4e-4fb5-b0df-7bfa50ed6477" path="/var/lib/kubelet/pods/896660eb-da4e-4fb5-b0df-7bfa50ed6477/volumes" Jul 20 16:48:05.608797 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:05.608732 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 16:48:05.609344 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:05.609082 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="alertmanager" containerID="cri-o://3ef446f24eb8277ea49a49440fea09bcfb45368efe1c427353fef9e49a04f0a1" gracePeriod=120 Jul 20 16:48:05.609344 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:05.609154 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="kube-rbac-proxy-metric" containerID="cri-o://7829bdcfa48f9bd3aa0cb7d75dddc147ca0d3c8bd392843c289ca1127d75c2b9" gracePeriod=120 Jul 20 16:48:05.609344 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:05.609189 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="config-reloader" containerID="cri-o://49dba45c7b7bc1ac91ee4879816558f398e80ee0dd9a0093eef86e4a7b5f4920" gracePeriod=120 Jul 20 16:48:05.609344 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:05.609213 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="kube-rbac-proxy" containerID="cri-o://64f3a05495e83161618ac79728c9b748384c8a090d9108d0dd5abcea3d06d686" gracePeriod=120 Jul 20 16:48:05.609344 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:05.609221 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="prom-label-proxy" containerID="cri-o://cc92ad4388386f1cb521161f43d8b97ffb7b545c96b910bf25b92621d665a13c" gracePeriod=120 Jul 20 16:48:05.609344 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:05.609165 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="kube-rbac-proxy-web" containerID="cri-o://590cafd2a18f14da1ee2ec104d701e2a728ca6f8b8170f666746c6b539f09f93" gracePeriod=120 Jul 20 16:48:06.543975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.543943 2566 generic.go:356] "Generic (PLEG): container finished" podID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerID="cc92ad4388386f1cb521161f43d8b97ffb7b545c96b910bf25b92621d665a13c" exitCode=0 Jul 20 16:48:06.543975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.543969 2566 generic.go:356] "Generic (PLEG): container finished" podID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerID="64f3a05495e83161618ac79728c9b748384c8a090d9108d0dd5abcea3d06d686" exitCode=0 Jul 20 16:48:06.543975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.543975 2566 generic.go:356] "Generic (PLEG): container finished" podID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerID="49dba45c7b7bc1ac91ee4879816558f398e80ee0dd9a0093eef86e4a7b5f4920" exitCode=0 Jul 20 16:48:06.543975 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.543981 2566 generic.go:356] "Generic (PLEG): container finished" podID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerID="3ef446f24eb8277ea49a49440fea09bcfb45368efe1c427353fef9e49a04f0a1" exitCode=0 Jul 20 16:48:06.544229 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.544011 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c49477a4-db75-4ac8-80fe-159ce55be73c","Type":"ContainerDied","Data":"cc92ad4388386f1cb521161f43d8b97ffb7b545c96b910bf25b92621d665a13c"} Jul 20 16:48:06.544229 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.544071 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c49477a4-db75-4ac8-80fe-159ce55be73c","Type":"ContainerDied","Data":"64f3a05495e83161618ac79728c9b748384c8a090d9108d0dd5abcea3d06d686"} Jul 20 16:48:06.544229 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.544099 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c49477a4-db75-4ac8-80fe-159ce55be73c","Type":"ContainerDied","Data":"49dba45c7b7bc1ac91ee4879816558f398e80ee0dd9a0093eef86e4a7b5f4920"} Jul 20 16:48:06.544229 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.544123 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c49477a4-db75-4ac8-80fe-159ce55be73c","Type":"ContainerDied","Data":"3ef446f24eb8277ea49a49440fea09bcfb45368efe1c427353fef9e49a04f0a1"} Jul 20 16:48:06.852021 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.852000 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:06.902698 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.902669 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"c49477a4-db75-4ac8-80fe-159ce55be73c\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " Jul 20 16:48:06.902875 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.902716 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vv5r6\" (UniqueName: \"kubernetes.io/projected/c49477a4-db75-4ac8-80fe-159ce55be73c-kube-api-access-vv5r6\") pod \"c49477a4-db75-4ac8-80fe-159ce55be73c\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " Jul 20 16:48:06.902875 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.902753 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-kube-rbac-proxy\") pod \"c49477a4-db75-4ac8-80fe-159ce55be73c\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " Jul 20 16:48:06.902875 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.902789 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c49477a4-db75-4ac8-80fe-159ce55be73c-tls-assets\") pod \"c49477a4-db75-4ac8-80fe-159ce55be73c\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " Jul 20 16:48:06.902875 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.902814 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-web-config\") pod \"c49477a4-db75-4ac8-80fe-159ce55be73c\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " Jul 20 16:48:06.902875 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.902851 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c49477a4-db75-4ac8-80fe-159ce55be73c-config-out\") pod \"c49477a4-db75-4ac8-80fe-159ce55be73c\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " Jul 20 16:48:06.903139 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.902877 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-kube-rbac-proxy-web\") pod \"c49477a4-db75-4ac8-80fe-159ce55be73c\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " Jul 20 16:48:06.903139 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.902903 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-main-tls\") pod \"c49477a4-db75-4ac8-80fe-159ce55be73c\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " Jul 20 16:48:06.903139 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.902941 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-cluster-tls-config\") pod \"c49477a4-db75-4ac8-80fe-159ce55be73c\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " Jul 20 16:48:06.903139 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.902976 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c49477a4-db75-4ac8-80fe-159ce55be73c-alertmanager-trusted-ca-bundle\") pod \"c49477a4-db75-4ac8-80fe-159ce55be73c\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " Jul 20 16:48:06.903139 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.903033 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-config-volume\") pod \"c49477a4-db75-4ac8-80fe-159ce55be73c\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " Jul 20 16:48:06.903139 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.903058 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c49477a4-db75-4ac8-80fe-159ce55be73c-alertmanager-main-db\") pod \"c49477a4-db75-4ac8-80fe-159ce55be73c\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " Jul 20 16:48:06.903139 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.903088 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c49477a4-db75-4ac8-80fe-159ce55be73c-metrics-client-ca\") pod \"c49477a4-db75-4ac8-80fe-159ce55be73c\" (UID: \"c49477a4-db75-4ac8-80fe-159ce55be73c\") " Jul 20 16:48:06.903696 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.903666 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c49477a4-db75-4ac8-80fe-159ce55be73c-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "c49477a4-db75-4ac8-80fe-159ce55be73c" (UID: "c49477a4-db75-4ac8-80fe-159ce55be73c"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:48:06.904518 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.904457 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c49477a4-db75-4ac8-80fe-159ce55be73c-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "c49477a4-db75-4ac8-80fe-159ce55be73c" (UID: "c49477a4-db75-4ac8-80fe-159ce55be73c"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:48:06.908332 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.908302 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "c49477a4-db75-4ac8-80fe-159ce55be73c" (UID: "c49477a4-db75-4ac8-80fe-159ce55be73c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:06.908954 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.908886 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c49477a4-db75-4ac8-80fe-159ce55be73c-kube-api-access-vv5r6" (OuterVolumeSpecName: "kube-api-access-vv5r6") pod "c49477a4-db75-4ac8-80fe-159ce55be73c" (UID: "c49477a4-db75-4ac8-80fe-159ce55be73c"). InnerVolumeSpecName "kube-api-access-vv5r6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:48:06.908954 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.908942 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "c49477a4-db75-4ac8-80fe-159ce55be73c" (UID: "c49477a4-db75-4ac8-80fe-159ce55be73c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:06.908954 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.908992 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c49477a4-db75-4ac8-80fe-159ce55be73c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c49477a4-db75-4ac8-80fe-159ce55be73c" (UID: "c49477a4-db75-4ac8-80fe-159ce55be73c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:48:06.909398 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.909230 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c49477a4-db75-4ac8-80fe-159ce55be73c-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "c49477a4-db75-4ac8-80fe-159ce55be73c" (UID: "c49477a4-db75-4ac8-80fe-159ce55be73c"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:48:06.909398 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.909322 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "c49477a4-db75-4ac8-80fe-159ce55be73c" (UID: "c49477a4-db75-4ac8-80fe-159ce55be73c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:06.909660 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.909627 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "c49477a4-db75-4ac8-80fe-159ce55be73c" (UID: "c49477a4-db75-4ac8-80fe-159ce55be73c"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:06.910480 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.910423 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-config-volume" (OuterVolumeSpecName: "config-volume") pod "c49477a4-db75-4ac8-80fe-159ce55be73c" (UID: "c49477a4-db75-4ac8-80fe-159ce55be73c"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:06.911642 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.911580 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c49477a4-db75-4ac8-80fe-159ce55be73c-config-out" (OuterVolumeSpecName: "config-out") pod "c49477a4-db75-4ac8-80fe-159ce55be73c" (UID: "c49477a4-db75-4ac8-80fe-159ce55be73c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:48:06.914253 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.914231 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "c49477a4-db75-4ac8-80fe-159ce55be73c" (UID: "c49477a4-db75-4ac8-80fe-159ce55be73c"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:06.922328 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:06.922304 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-web-config" (OuterVolumeSpecName: "web-config") pod "c49477a4-db75-4ac8-80fe-159ce55be73c" (UID: "c49477a4-db75-4ac8-80fe-159ce55be73c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:07.004445 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.004413 2566 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-cluster-tls-config\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:07.004445 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.004444 2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c49477a4-db75-4ac8-80fe-159ce55be73c-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:07.004638 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.004455 2566 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-config-volume\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:07.004638 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.004464 2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c49477a4-db75-4ac8-80fe-159ce55be73c-alertmanager-main-db\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:07.004638 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.004474 2566 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c49477a4-db75-4ac8-80fe-159ce55be73c-metrics-client-ca\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:07.004638 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.004485 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:07.004638 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.004494 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv5r6\" (UniqueName: \"kubernetes.io/projected/c49477a4-db75-4ac8-80fe-159ce55be73c-kube-api-access-vv5r6\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:07.004638 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.004502 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:07.004638 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.004510 2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c49477a4-db75-4ac8-80fe-159ce55be73c-tls-assets\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:07.004638 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.004519 2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-web-config\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:07.004638 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.004526 2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c49477a4-db75-4ac8-80fe-159ce55be73c-config-out\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:07.004638 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.004534 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:07.004638 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.004543 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c49477a4-db75-4ac8-80fe-159ce55be73c-secret-alertmanager-main-tls\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:07.551959 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.551924 2566 generic.go:356] "Generic (PLEG): container finished" podID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerID="7829bdcfa48f9bd3aa0cb7d75dddc147ca0d3c8bd392843c289ca1127d75c2b9" exitCode=0 Jul 20 16:48:07.551959 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.551950 2566 generic.go:356] "Generic (PLEG): container finished" podID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerID="590cafd2a18f14da1ee2ec104d701e2a728ca6f8b8170f666746c6b539f09f93" exitCode=0 Jul 20 16:48:07.552158 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.551982 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c49477a4-db75-4ac8-80fe-159ce55be73c","Type":"ContainerDied","Data":"7829bdcfa48f9bd3aa0cb7d75dddc147ca0d3c8bd392843c289ca1127d75c2b9"} Jul 20 16:48:07.552158 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.552000 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.552158 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.552046 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c49477a4-db75-4ac8-80fe-159ce55be73c","Type":"ContainerDied","Data":"590cafd2a18f14da1ee2ec104d701e2a728ca6f8b8170f666746c6b539f09f93"} Jul 20 16:48:07.552158 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.552074 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c49477a4-db75-4ac8-80fe-159ce55be73c","Type":"ContainerDied","Data":"2062556e02b331162553cd8e427da451de841bf59c459e13bd403e7419d068c7"} Jul 20 16:48:07.552158 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.552115 2566 scope.go:117] "RemoveContainer" containerID="cc92ad4388386f1cb521161f43d8b97ffb7b545c96b910bf25b92621d665a13c" Jul 20 16:48:07.560264 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.560240 2566 scope.go:117] "RemoveContainer" containerID="7829bdcfa48f9bd3aa0cb7d75dddc147ca0d3c8bd392843c289ca1127d75c2b9" Jul 20 16:48:07.568436 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.568413 2566 scope.go:117] "RemoveContainer" containerID="64f3a05495e83161618ac79728c9b748384c8a090d9108d0dd5abcea3d06d686" Jul 20 16:48:07.575101 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.575077 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 16:48:07.575483 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.575466 2566 scope.go:117] "RemoveContainer" containerID="590cafd2a18f14da1ee2ec104d701e2a728ca6f8b8170f666746c6b539f09f93" Jul 20 16:48:07.579650 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.579629 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 16:48:07.582853 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.582835 2566 scope.go:117] "RemoveContainer" containerID="49dba45c7b7bc1ac91ee4879816558f398e80ee0dd9a0093eef86e4a7b5f4920" Jul 20 16:48:07.589920 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.589904 2566 scope.go:117] "RemoveContainer" containerID="3ef446f24eb8277ea49a49440fea09bcfb45368efe1c427353fef9e49a04f0a1" Jul 20 16:48:07.596913 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.596895 2566 scope.go:117] "RemoveContainer" containerID="9bb14fe6c03605c75bcafd042d14d38d4064d611efd7f51212a52e428025fd03" Jul 20 16:48:07.603305 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.603277 2566 scope.go:117] "RemoveContainer" containerID="cc92ad4388386f1cb521161f43d8b97ffb7b545c96b910bf25b92621d665a13c" Jul 20 16:48:07.603562 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:48:07.603541 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc92ad4388386f1cb521161f43d8b97ffb7b545c96b910bf25b92621d665a13c\": container with ID starting with cc92ad4388386f1cb521161f43d8b97ffb7b545c96b910bf25b92621d665a13c not found: ID does not exist" containerID="cc92ad4388386f1cb521161f43d8b97ffb7b545c96b910bf25b92621d665a13c" Jul 20 16:48:07.603637 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.603569 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc92ad4388386f1cb521161f43d8b97ffb7b545c96b910bf25b92621d665a13c"} err="failed to get container status \"cc92ad4388386f1cb521161f43d8b97ffb7b545c96b910bf25b92621d665a13c\": rpc error: code = NotFound desc = could not find container \"cc92ad4388386f1cb521161f43d8b97ffb7b545c96b910bf25b92621d665a13c\": container with ID starting with cc92ad4388386f1cb521161f43d8b97ffb7b545c96b910bf25b92621d665a13c not found: ID does not exist" Jul 20 16:48:07.603637 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.603588 2566 scope.go:117] "RemoveContainer" containerID="7829bdcfa48f9bd3aa0cb7d75dddc147ca0d3c8bd392843c289ca1127d75c2b9" Jul 20 16:48:07.603874 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:48:07.603851 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7829bdcfa48f9bd3aa0cb7d75dddc147ca0d3c8bd392843c289ca1127d75c2b9\": container with ID starting with 7829bdcfa48f9bd3aa0cb7d75dddc147ca0d3c8bd392843c289ca1127d75c2b9 not found: ID does not exist" containerID="7829bdcfa48f9bd3aa0cb7d75dddc147ca0d3c8bd392843c289ca1127d75c2b9" Jul 20 16:48:07.603943 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.603885 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7829bdcfa48f9bd3aa0cb7d75dddc147ca0d3c8bd392843c289ca1127d75c2b9"} err="failed to get container status \"7829bdcfa48f9bd3aa0cb7d75dddc147ca0d3c8bd392843c289ca1127d75c2b9\": rpc error: code = NotFound desc = could not find container \"7829bdcfa48f9bd3aa0cb7d75dddc147ca0d3c8bd392843c289ca1127d75c2b9\": container with ID starting with 7829bdcfa48f9bd3aa0cb7d75dddc147ca0d3c8bd392843c289ca1127d75c2b9 not found: ID does not exist" Jul 20 16:48:07.603943 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.603905 2566 scope.go:117] "RemoveContainer" containerID="64f3a05495e83161618ac79728c9b748384c8a090d9108d0dd5abcea3d06d686" Jul 20 16:48:07.604148 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:48:07.604134 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64f3a05495e83161618ac79728c9b748384c8a090d9108d0dd5abcea3d06d686\": container with ID starting with 64f3a05495e83161618ac79728c9b748384c8a090d9108d0dd5abcea3d06d686 not found: ID does not exist" containerID="64f3a05495e83161618ac79728c9b748384c8a090d9108d0dd5abcea3d06d686" Jul 20 16:48:07.604194 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.604150 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64f3a05495e83161618ac79728c9b748384c8a090d9108d0dd5abcea3d06d686"} err="failed to get container status \"64f3a05495e83161618ac79728c9b748384c8a090d9108d0dd5abcea3d06d686\": rpc error: code = NotFound desc = could not find container \"64f3a05495e83161618ac79728c9b748384c8a090d9108d0dd5abcea3d06d686\": container with ID starting with 64f3a05495e83161618ac79728c9b748384c8a090d9108d0dd5abcea3d06d686 not found: ID does not exist" Jul 20 16:48:07.604194 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.604161 2566 scope.go:117] "RemoveContainer" containerID="590cafd2a18f14da1ee2ec104d701e2a728ca6f8b8170f666746c6b539f09f93" Jul 20 16:48:07.604356 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:48:07.604343 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"590cafd2a18f14da1ee2ec104d701e2a728ca6f8b8170f666746c6b539f09f93\": container with ID starting with 590cafd2a18f14da1ee2ec104d701e2a728ca6f8b8170f666746c6b539f09f93 not found: ID does not exist" containerID="590cafd2a18f14da1ee2ec104d701e2a728ca6f8b8170f666746c6b539f09f93" Jul 20 16:48:07.604413 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.604358 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"590cafd2a18f14da1ee2ec104d701e2a728ca6f8b8170f666746c6b539f09f93"} err="failed to get container status \"590cafd2a18f14da1ee2ec104d701e2a728ca6f8b8170f666746c6b539f09f93\": rpc error: code = NotFound desc = could not find container \"590cafd2a18f14da1ee2ec104d701e2a728ca6f8b8170f666746c6b539f09f93\": container with ID starting with 590cafd2a18f14da1ee2ec104d701e2a728ca6f8b8170f666746c6b539f09f93 not found: ID does not exist" Jul 20 16:48:07.604454 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.604432 2566 scope.go:117] "RemoveContainer" containerID="49dba45c7b7bc1ac91ee4879816558f398e80ee0dd9a0093eef86e4a7b5f4920" Jul 20 16:48:07.604704 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:48:07.604682 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49dba45c7b7bc1ac91ee4879816558f398e80ee0dd9a0093eef86e4a7b5f4920\": container with ID starting with 49dba45c7b7bc1ac91ee4879816558f398e80ee0dd9a0093eef86e4a7b5f4920 not found: ID does not exist" containerID="49dba45c7b7bc1ac91ee4879816558f398e80ee0dd9a0093eef86e4a7b5f4920" Jul 20 16:48:07.604760 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.604713 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49dba45c7b7bc1ac91ee4879816558f398e80ee0dd9a0093eef86e4a7b5f4920"} err="failed to get container status \"49dba45c7b7bc1ac91ee4879816558f398e80ee0dd9a0093eef86e4a7b5f4920\": rpc error: code = NotFound desc = could not find container \"49dba45c7b7bc1ac91ee4879816558f398e80ee0dd9a0093eef86e4a7b5f4920\": container with ID starting with 49dba45c7b7bc1ac91ee4879816558f398e80ee0dd9a0093eef86e4a7b5f4920 not found: ID does not exist" Jul 20 16:48:07.604760 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.604732 2566 scope.go:117] "RemoveContainer" containerID="3ef446f24eb8277ea49a49440fea09bcfb45368efe1c427353fef9e49a04f0a1" Jul 20 16:48:07.605231 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:48:07.605210 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ef446f24eb8277ea49a49440fea09bcfb45368efe1c427353fef9e49a04f0a1\": container with ID starting with 3ef446f24eb8277ea49a49440fea09bcfb45368efe1c427353fef9e49a04f0a1 not found: ID does not exist" containerID="3ef446f24eb8277ea49a49440fea09bcfb45368efe1c427353fef9e49a04f0a1" Jul 20 16:48:07.605310 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.605235 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ef446f24eb8277ea49a49440fea09bcfb45368efe1c427353fef9e49a04f0a1"} err="failed to get container status \"3ef446f24eb8277ea49a49440fea09bcfb45368efe1c427353fef9e49a04f0a1\": rpc error: code = NotFound desc = could not find container \"3ef446f24eb8277ea49a49440fea09bcfb45368efe1c427353fef9e49a04f0a1\": container with ID starting with 3ef446f24eb8277ea49a49440fea09bcfb45368efe1c427353fef9e49a04f0a1 not found: ID does not exist" Jul 20 16:48:07.605310 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.605250 2566 scope.go:117] "RemoveContainer" containerID="9bb14fe6c03605c75bcafd042d14d38d4064d611efd7f51212a52e428025fd03" Jul 20 16:48:07.605509 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:48:07.605494 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bb14fe6c03605c75bcafd042d14d38d4064d611efd7f51212a52e428025fd03\": container with ID starting with 9bb14fe6c03605c75bcafd042d14d38d4064d611efd7f51212a52e428025fd03 not found: ID does not exist" containerID="9bb14fe6c03605c75bcafd042d14d38d4064d611efd7f51212a52e428025fd03" Jul 20 16:48:07.605550 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.605515 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bb14fe6c03605c75bcafd042d14d38d4064d611efd7f51212a52e428025fd03"} err="failed to get container status \"9bb14fe6c03605c75bcafd042d14d38d4064d611efd7f51212a52e428025fd03\": rpc error: code = NotFound desc = could not find container \"9bb14fe6c03605c75bcafd042d14d38d4064d611efd7f51212a52e428025fd03\": container with ID starting with 9bb14fe6c03605c75bcafd042d14d38d4064d611efd7f51212a52e428025fd03 not found: ID does not exist" Jul 20 16:48:07.605550 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.605530 2566 scope.go:117] "RemoveContainer" containerID="cc92ad4388386f1cb521161f43d8b97ffb7b545c96b910bf25b92621d665a13c" Jul 20 16:48:07.605748 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.605730 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc92ad4388386f1cb521161f43d8b97ffb7b545c96b910bf25b92621d665a13c"} err="failed to get container status \"cc92ad4388386f1cb521161f43d8b97ffb7b545c96b910bf25b92621d665a13c\": rpc error: code = NotFound desc = could not find container \"cc92ad4388386f1cb521161f43d8b97ffb7b545c96b910bf25b92621d665a13c\": container with ID starting with cc92ad4388386f1cb521161f43d8b97ffb7b545c96b910bf25b92621d665a13c not found: ID does not exist" Jul 20 16:48:07.605813 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.605748 2566 scope.go:117] "RemoveContainer" containerID="7829bdcfa48f9bd3aa0cb7d75dddc147ca0d3c8bd392843c289ca1127d75c2b9" Jul 20 16:48:07.605997 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.605981 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7829bdcfa48f9bd3aa0cb7d75dddc147ca0d3c8bd392843c289ca1127d75c2b9"} err="failed to get container status \"7829bdcfa48f9bd3aa0cb7d75dddc147ca0d3c8bd392843c289ca1127d75c2b9\": rpc error: code = NotFound desc = could not find container \"7829bdcfa48f9bd3aa0cb7d75dddc147ca0d3c8bd392843c289ca1127d75c2b9\": container with ID starting with 7829bdcfa48f9bd3aa0cb7d75dddc147ca0d3c8bd392843c289ca1127d75c2b9 not found: ID does not exist" Jul 20 16:48:07.606043 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.606008 2566 scope.go:117] "RemoveContainer" containerID="64f3a05495e83161618ac79728c9b748384c8a090d9108d0dd5abcea3d06d686" Jul 20 16:48:07.606215 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.606201 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64f3a05495e83161618ac79728c9b748384c8a090d9108d0dd5abcea3d06d686"} err="failed to get container status \"64f3a05495e83161618ac79728c9b748384c8a090d9108d0dd5abcea3d06d686\": rpc error: code = NotFound desc = could not find container \"64f3a05495e83161618ac79728c9b748384c8a090d9108d0dd5abcea3d06d686\": container with ID starting with 64f3a05495e83161618ac79728c9b748384c8a090d9108d0dd5abcea3d06d686 not found: ID does not exist" Jul 20 16:48:07.606263 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.606216 2566 scope.go:117] "RemoveContainer" containerID="590cafd2a18f14da1ee2ec104d701e2a728ca6f8b8170f666746c6b539f09f93" Jul 20 16:48:07.606390 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.606375 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"590cafd2a18f14da1ee2ec104d701e2a728ca6f8b8170f666746c6b539f09f93"} err="failed to get container status \"590cafd2a18f14da1ee2ec104d701e2a728ca6f8b8170f666746c6b539f09f93\": rpc error: code = NotFound desc = could not find container \"590cafd2a18f14da1ee2ec104d701e2a728ca6f8b8170f666746c6b539f09f93\": container with ID starting with 590cafd2a18f14da1ee2ec104d701e2a728ca6f8b8170f666746c6b539f09f93 not found: ID does not exist" Jul 20 16:48:07.606435 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.606390 2566 scope.go:117] "RemoveContainer" containerID="49dba45c7b7bc1ac91ee4879816558f398e80ee0dd9a0093eef86e4a7b5f4920" Jul 20 16:48:07.606586 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.606570 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49dba45c7b7bc1ac91ee4879816558f398e80ee0dd9a0093eef86e4a7b5f4920"} err="failed to get container status \"49dba45c7b7bc1ac91ee4879816558f398e80ee0dd9a0093eef86e4a7b5f4920\": rpc error: code = NotFound desc = could not find container \"49dba45c7b7bc1ac91ee4879816558f398e80ee0dd9a0093eef86e4a7b5f4920\": container with ID starting with 49dba45c7b7bc1ac91ee4879816558f398e80ee0dd9a0093eef86e4a7b5f4920 not found: ID does not exist" Jul 20 16:48:07.606630 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.606587 2566 scope.go:117] "RemoveContainer" containerID="3ef446f24eb8277ea49a49440fea09bcfb45368efe1c427353fef9e49a04f0a1" Jul 20 16:48:07.606820 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.606798 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ef446f24eb8277ea49a49440fea09bcfb45368efe1c427353fef9e49a04f0a1"} err="failed to get container status \"3ef446f24eb8277ea49a49440fea09bcfb45368efe1c427353fef9e49a04f0a1\": rpc error: code = NotFound desc = could not find container \"3ef446f24eb8277ea49a49440fea09bcfb45368efe1c427353fef9e49a04f0a1\": container with ID starting with 3ef446f24eb8277ea49a49440fea09bcfb45368efe1c427353fef9e49a04f0a1 not found: ID does not exist" Jul 20 16:48:07.606820 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.606820 2566 scope.go:117] "RemoveContainer" containerID="9bb14fe6c03605c75bcafd042d14d38d4064d611efd7f51212a52e428025fd03" Jul 20 16:48:07.607023 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.607000 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bb14fe6c03605c75bcafd042d14d38d4064d611efd7f51212a52e428025fd03"} err="failed to get container status \"9bb14fe6c03605c75bcafd042d14d38d4064d611efd7f51212a52e428025fd03\": rpc error: code = NotFound desc = could not find container \"9bb14fe6c03605c75bcafd042d14d38d4064d611efd7f51212a52e428025fd03\": container with ID starting with 9bb14fe6c03605c75bcafd042d14d38d4064d611efd7f51212a52e428025fd03 not found: ID does not exist" Jul 20 16:48:07.619303 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619281 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 16:48:07.619591 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619578 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="kube-rbac-proxy" Jul 20 16:48:07.619630 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619593 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="kube-rbac-proxy" Jul 20 16:48:07.619630 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619601 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="kube-rbac-proxy-web" Jul 20 16:48:07.619630 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619607 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="kube-rbac-proxy-web" Jul 20 16:48:07.619630 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619617 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="init-config-reloader" Jul 20 16:48:07.619630 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619623 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="init-config-reloader" Jul 20 16:48:07.619766 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619633 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="alertmanager" Jul 20 16:48:07.619766 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619642 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="alertmanager" Jul 20 16:48:07.619766 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619653 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f35f1455-d7b5-4e8f-99e6-841c4d3c30fd" containerName="console" Jul 20 16:48:07.619766 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619658 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f35f1455-d7b5-4e8f-99e6-841c4d3c30fd" containerName="console" Jul 20 16:48:07.619766 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619666 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="896660eb-da4e-4fb5-b0df-7bfa50ed6477" containerName="console" Jul 20 16:48:07.619766 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619671 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="896660eb-da4e-4fb5-b0df-7bfa50ed6477" containerName="console" Jul 20 16:48:07.619766 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619676 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="config-reloader" Jul 20 16:48:07.619766 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619681 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="config-reloader" Jul 20 16:48:07.619766 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619689 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="prom-label-proxy" Jul 20 16:48:07.619766 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619694 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="prom-label-proxy" Jul 20 16:48:07.619766 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619700 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00742c8a-f4c0-4804-a13e-254428723a76" containerName="console" Jul 20 16:48:07.619766 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619706 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="00742c8a-f4c0-4804-a13e-254428723a76" containerName="console" Jul 20 16:48:07.619766 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619717 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="kube-rbac-proxy-metric" Jul 20 16:48:07.619766 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619725 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="kube-rbac-proxy-metric" Jul 20 16:48:07.620189 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619801 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f35f1455-d7b5-4e8f-99e6-841c4d3c30fd" containerName="console" Jul 20 16:48:07.620189 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619812 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="alertmanager" Jul 20 16:48:07.620189 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619817 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="config-reloader" Jul 20 16:48:07.620189 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619824 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="896660eb-da4e-4fb5-b0df-7bfa50ed6477" containerName="console" Jul 20 16:48:07.620189 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619830 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="prom-label-proxy" Jul 20 16:48:07.620189 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619835 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="kube-rbac-proxy" Jul 20 16:48:07.620189 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619842 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="kube-rbac-proxy-web" Jul 20 16:48:07.620189 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619847 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="00742c8a-f4c0-4804-a13e-254428723a76" containerName="console" Jul 20 16:48:07.620189 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.619852 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" containerName="kube-rbac-proxy-metric" Jul 20 16:48:07.623208 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.623193 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.625538 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.625522 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 20 16:48:07.625662 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.625574 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 20 16:48:07.627228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.627193 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 20 16:48:07.627425 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.627404 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 20 16:48:07.627501 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.627411 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 20 16:48:07.627895 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.627881 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bvm75\"" Jul 20 16:48:07.628339 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.628324 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 20 16:48:07.628470 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.628404 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 20 16:48:07.628470 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.628412 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 20 16:48:07.636187 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.636162 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 20 16:48:07.671752 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.671719 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 16:48:07.711251 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.711224 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.711384 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.711257 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxlf2\" (UniqueName: \"kubernetes.io/projected/4a5fd763-c746-41c4-988d-894a46a5e08b-kube-api-access-rxlf2\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.711384 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.711279 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a5fd763-c746-41c4-988d-894a46a5e08b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.711384 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.711294 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a5fd763-c746-41c4-988d-894a46a5e08b-config-out\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.711384 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.711339 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.711384 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.711372 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a5fd763-c746-41c4-988d-894a46a5e08b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.711606 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.711398 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4a5fd763-c746-41c4-988d-894a46a5e08b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.711606 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.711439 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.711606 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.711464 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.711606 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.711480 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.711606 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.711499 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-config-volume\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.711606 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.711579 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a5fd763-c746-41c4-988d-894a46a5e08b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.711843 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.711650 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-web-config\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.812111 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.812035 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a5fd763-c746-41c4-988d-894a46a5e08b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.812111 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.812091 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-web-config\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.812319 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.812140 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.812319 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.812170 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxlf2\" (UniqueName: \"kubernetes.io/projected/4a5fd763-c746-41c4-988d-894a46a5e08b-kube-api-access-rxlf2\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.812319 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.812217 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a5fd763-c746-41c4-988d-894a46a5e08b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.812319 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.812249 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a5fd763-c746-41c4-988d-894a46a5e08b-config-out\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.812319 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.812289 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.812554 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.812322 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a5fd763-c746-41c4-988d-894a46a5e08b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.812554 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.812352 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4a5fd763-c746-41c4-988d-894a46a5e08b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.812554 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.812398 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.812554 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.812429 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.812554 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.812453 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.812554 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.812482 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-config-volume\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.812932 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.812839 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a5fd763-c746-41c4-988d-894a46a5e08b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.812932 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.812882 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4a5fd763-c746-41c4-988d-894a46a5e08b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.814631 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.814594 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a5fd763-c746-41c4-988d-894a46a5e08b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.815062 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.815037 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c49477a4-db75-4ac8-80fe-159ce55be73c" path="/var/lib/kubelet/pods/c49477a4-db75-4ac8-80fe-159ce55be73c/volumes" Jul 20 16:48:07.816911 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.816249 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.816911 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.816654 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a5fd763-c746-41c4-988d-894a46a5e08b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.816911 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.816823 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-web-config\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.817474 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.817106 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.817474 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.817458 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.817620 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.817562 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-config-volume\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.818012 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.817992 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a5fd763-c746-41c4-988d-894a46a5e08b-config-out\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.818339 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.818319 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.818484 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.818467 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a5fd763-c746-41c4-988d-894a46a5e08b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.822295 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.822277 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxlf2\" (UniqueName: \"kubernetes.io/projected/4a5fd763-c746-41c4-988d-894a46a5e08b-kube-api-access-rxlf2\") pod \"alertmanager-main-0\" (UID: \"4a5fd763-c746-41c4-988d-894a46a5e08b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:07.933510 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:07.933478 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 16:48:08.076622 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:08.076601 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 16:48:08.078491 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:48:08.078464 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a5fd763_c746_41c4_988d_894a46a5e08b.slice/crio-f415da6ca8fca0903714029fbb12701f706fd41abb95abb21deec9bd92eefdd8 WatchSource:0}: Error finding container f415da6ca8fca0903714029fbb12701f706fd41abb95abb21deec9bd92eefdd8: Status 404 returned error can't find the container with id f415da6ca8fca0903714029fbb12701f706fd41abb95abb21deec9bd92eefdd8 Jul 20 16:48:08.558244 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:08.558215 2566 generic.go:356] "Generic (PLEG): container finished" podID="4a5fd763-c746-41c4-988d-894a46a5e08b" containerID="d6eeed1c2e973292f171a40f2f938ebb76a0c215bfd93c340c710f1727642f87" exitCode=0 Jul 20 16:48:08.558414 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:08.558281 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a5fd763-c746-41c4-988d-894a46a5e08b","Type":"ContainerDied","Data":"d6eeed1c2e973292f171a40f2f938ebb76a0c215bfd93c340c710f1727642f87"} Jul 20 16:48:08.558414 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:08.558330 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a5fd763-c746-41c4-988d-894a46a5e08b","Type":"ContainerStarted","Data":"f415da6ca8fca0903714029fbb12701f706fd41abb95abb21deec9bd92eefdd8"} Jul 20 16:48:09.566216 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:09.566183 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a5fd763-c746-41c4-988d-894a46a5e08b","Type":"ContainerStarted","Data":"992b6b6b395e2070cb16ad598239fa6c97f32d9f19dfa558a7490a90933c3d40"} Jul 20 16:48:09.566629 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:09.566255 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a5fd763-c746-41c4-988d-894a46a5e08b","Type":"ContainerStarted","Data":"2aef270ecd9ce46104e1c6555a9ffe492efeaf178fff22ba185c5ece55fddd62"} Jul 20 16:48:09.566629 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:09.566281 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a5fd763-c746-41c4-988d-894a46a5e08b","Type":"ContainerStarted","Data":"e797a8a1b00e0d3af987ae88d99dd074614558c6cb98d156b227b5fda07984a3"} Jul 20 16:48:09.566629 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:09.566307 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a5fd763-c746-41c4-988d-894a46a5e08b","Type":"ContainerStarted","Data":"fa36fbf7c159eb9413533fb86b13fd720156c6e592cd08e0fb2c1b7d072ed499"} Jul 20 16:48:09.566629 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:09.566340 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a5fd763-c746-41c4-988d-894a46a5e08b","Type":"ContainerStarted","Data":"b42e5bef4ef104e3d51b48602277d21ace4116e20f60612fa9cd3d7950768245"} Jul 20 16:48:09.566629 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:09.566367 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a5fd763-c746-41c4-988d-894a46a5e08b","Type":"ContainerStarted","Data":"44b45f29715fff685a07eaca5d2b7fa4ba85aff056d66321f274230e363cf3da"} Jul 20 16:48:09.593561 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:09.593511 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.577484465 podStartE2EDuration="2.593496498s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.012033ms" isStatefulPod=false podCreationTimestamp="2026-07-20 16:48:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:48:09.592078042 +0000 UTC m=+154.378077122" watchObservedRunningTime="2026-07-20 16:48:09.593496498 +0000 UTC m=+154.379495577" Jul 20 16:48:22.124187 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.124145 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c8b758c77-gmbmn"] Jul 20 16:48:22.127228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.127204 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.141832 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.141808 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c8b758c77-gmbmn"] Jul 20 16:48:22.228339 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.228303 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d872c757-66fb-4b42-a5eb-a1660f7834f6-console-serving-cert\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.228339 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.228335 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-console-config\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.228528 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.228355 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d872c757-66fb-4b42-a5eb-a1660f7834f6-console-oauth-config\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.228528 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.228382 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-service-ca\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.228528 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.228446 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-trusted-ca-bundle\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.228528 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.228480 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-oauth-serving-cert\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.228528 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.228495 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmfh4\" (UniqueName: \"kubernetes.io/projected/d872c757-66fb-4b42-a5eb-a1660f7834f6-kube-api-access-lmfh4\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.329348 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.329315 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d872c757-66fb-4b42-a5eb-a1660f7834f6-console-serving-cert\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.329491 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.329364 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-console-config\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.329491 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.329410 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d872c757-66fb-4b42-a5eb-a1660f7834f6-console-oauth-config\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.329491 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.329452 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-service-ca\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.329491 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.329472 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-trusted-ca-bundle\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.329491 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.329490 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-oauth-serving-cert\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.329734 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.329506 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmfh4\" (UniqueName: \"kubernetes.io/projected/d872c757-66fb-4b42-a5eb-a1660f7834f6-kube-api-access-lmfh4\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.330144 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.330124 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-console-config\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.330238 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.330199 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-service-ca\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.330238 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.330227 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-oauth-serving-cert\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.330637 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.330616 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-trusted-ca-bundle\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.332230 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.332207 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d872c757-66fb-4b42-a5eb-a1660f7834f6-console-oauth-config\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.332275 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.332218 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d872c757-66fb-4b42-a5eb-a1660f7834f6-console-serving-cert\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.339147 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.339128 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmfh4\" (UniqueName: \"kubernetes.io/projected/d872c757-66fb-4b42-a5eb-a1660f7834f6-kube-api-access-lmfh4\") pod \"console-5c8b758c77-gmbmn\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.437311 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.437248 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:22.569112 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.569019 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c8b758c77-gmbmn"] Jul 20 16:48:22.571616 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:48:22.571587 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd872c757_66fb_4b42_a5eb_a1660f7834f6.slice/crio-0443fe43746acba0def277196bf6a3dfd3bc7606b06f216e4e2d467aaad5e07a WatchSource:0}: Error finding container 0443fe43746acba0def277196bf6a3dfd3bc7606b06f216e4e2d467aaad5e07a: Status 404 returned error can't find the container with id 0443fe43746acba0def277196bf6a3dfd3bc7606b06f216e4e2d467aaad5e07a Jul 20 16:48:22.626900 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:22.626869 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c8b758c77-gmbmn" event={"ID":"d872c757-66fb-4b42-a5eb-a1660f7834f6","Type":"ContainerStarted","Data":"0443fe43746acba0def277196bf6a3dfd3bc7606b06f216e4e2d467aaad5e07a"} Jul 20 16:48:23.632906 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:23.632864 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c8b758c77-gmbmn" event={"ID":"d872c757-66fb-4b42-a5eb-a1660f7834f6","Type":"ContainerStarted","Data":"40ccb96266de98be63740646bb6a655b511c82cb9b12dc2e01dae447ee67cbfa"} Jul 20 16:48:23.651497 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:23.651436 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c8b758c77-gmbmn" podStartSLOduration=1.65141626 podStartE2EDuration="1.65141626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:48:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:48:23.651050977 +0000 UTC m=+168.437050056" watchObservedRunningTime="2026-07-20 16:48:23.65141626 +0000 UTC m=+168.437415346" Jul 20 16:48:32.438021 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:32.437920 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:32.438021 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:32.438007 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:32.442702 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:32.442678 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:32.675345 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:32.675319 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:48:32.714104 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:32.714017 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68ffc7f898-j4bwj"] Jul 20 16:48:57.737205 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:57.737142 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-68ffc7f898-j4bwj" podUID="308ef8a5-3aaf-42c5-b2b3-56a044947a8e" containerName="console" containerID="cri-o://194c7510665dc941a989d821f329ab2fd428187c0d332bfb142ed22106429bc3" gracePeriod=15 Jul 20 16:48:57.976052 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:57.976025 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:48:58.058369 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.058282 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-console-config\") pod \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " Jul 20 16:48:58.058369 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.058330 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-console-serving-cert\") pod \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " Jul 20 16:48:58.058369 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.058355 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x59dt\" (UniqueName: \"kubernetes.io/projected/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-kube-api-access-x59dt\") pod \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " Jul 20 16:48:58.058369 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.058371 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-oauth-serving-cert\") pod \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " Jul 20 16:48:58.058699 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.058397 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-console-oauth-config\") pod \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " Jul 20 16:48:58.058699 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.058421 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-service-ca\") pod \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " Jul 20 16:48:58.058699 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.058459 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-trusted-ca-bundle\") pod \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\" (UID: \"308ef8a5-3aaf-42c5-b2b3-56a044947a8e\") " Jul 20 16:48:58.058944 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.058728 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-console-config" (OuterVolumeSpecName: "console-config") pod "308ef8a5-3aaf-42c5-b2b3-56a044947a8e" (UID: "308ef8a5-3aaf-42c5-b2b3-56a044947a8e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:48:58.058944 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.058891 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "308ef8a5-3aaf-42c5-b2b3-56a044947a8e" (UID: "308ef8a5-3aaf-42c5-b2b3-56a044947a8e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:48:58.058944 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.058918 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-service-ca" (OuterVolumeSpecName: "service-ca") pod "308ef8a5-3aaf-42c5-b2b3-56a044947a8e" (UID: "308ef8a5-3aaf-42c5-b2b3-56a044947a8e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:48:58.059048 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.058995 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "308ef8a5-3aaf-42c5-b2b3-56a044947a8e" (UID: "308ef8a5-3aaf-42c5-b2b3-56a044947a8e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:48:58.060754 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.060725 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "308ef8a5-3aaf-42c5-b2b3-56a044947a8e" (UID: "308ef8a5-3aaf-42c5-b2b3-56a044947a8e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:58.061172 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.061143 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-kube-api-access-x59dt" (OuterVolumeSpecName: "kube-api-access-x59dt") pod "308ef8a5-3aaf-42c5-b2b3-56a044947a8e" (UID: "308ef8a5-3aaf-42c5-b2b3-56a044947a8e"). InnerVolumeSpecName "kube-api-access-x59dt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:48:58.061172 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.061153 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "308ef8a5-3aaf-42c5-b2b3-56a044947a8e" (UID: "308ef8a5-3aaf-42c5-b2b3-56a044947a8e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:58.159903 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.159871 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-console-config\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:58.159903 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.159896 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-console-serving-cert\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:58.159903 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.159906 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x59dt\" (UniqueName: \"kubernetes.io/projected/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-kube-api-access-x59dt\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:58.160126 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.159915 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-oauth-serving-cert\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:58.160126 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.159926 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-console-oauth-config\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:58.160126 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.159935 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-service-ca\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:58.160126 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.159942 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/308ef8a5-3aaf-42c5-b2b3-56a044947a8e-trusted-ca-bundle\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:48:58.777047 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.777016 2566 generic.go:356] "Generic (PLEG): container finished" podID="308ef8a5-3aaf-42c5-b2b3-56a044947a8e" containerID="194c7510665dc941a989d821f329ab2fd428187c0d332bfb142ed22106429bc3" exitCode=0 Jul 20 16:48:58.777491 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.777064 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68ffc7f898-j4bwj" Jul 20 16:48:58.777491 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.777073 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68ffc7f898-j4bwj" event={"ID":"308ef8a5-3aaf-42c5-b2b3-56a044947a8e","Type":"ContainerDied","Data":"194c7510665dc941a989d821f329ab2fd428187c0d332bfb142ed22106429bc3"} Jul 20 16:48:58.777491 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.777114 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68ffc7f898-j4bwj" event={"ID":"308ef8a5-3aaf-42c5-b2b3-56a044947a8e","Type":"ContainerDied","Data":"48f9ae395c8167c078341dba2d717d5a2450437982cb15feacd3abd32a126ca1"} Jul 20 16:48:58.777491 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.777133 2566 scope.go:117] "RemoveContainer" containerID="194c7510665dc941a989d821f329ab2fd428187c0d332bfb142ed22106429bc3" Jul 20 16:48:58.786306 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.786285 2566 scope.go:117] "RemoveContainer" containerID="194c7510665dc941a989d821f329ab2fd428187c0d332bfb142ed22106429bc3" Jul 20 16:48:58.786627 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:48:58.786605 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"194c7510665dc941a989d821f329ab2fd428187c0d332bfb142ed22106429bc3\": container with ID starting with 194c7510665dc941a989d821f329ab2fd428187c0d332bfb142ed22106429bc3 not found: ID does not exist" containerID="194c7510665dc941a989d821f329ab2fd428187c0d332bfb142ed22106429bc3" Jul 20 16:48:58.786675 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.786642 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"194c7510665dc941a989d821f329ab2fd428187c0d332bfb142ed22106429bc3"} err="failed to get container status \"194c7510665dc941a989d821f329ab2fd428187c0d332bfb142ed22106429bc3\": rpc error: code = NotFound desc = could not find container \"194c7510665dc941a989d821f329ab2fd428187c0d332bfb142ed22106429bc3\": container with ID starting with 194c7510665dc941a989d821f329ab2fd428187c0d332bfb142ed22106429bc3 not found: ID does not exist" Jul 20 16:48:58.802994 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.802965 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68ffc7f898-j4bwj"] Jul 20 16:48:58.811492 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:58.811467 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68ffc7f898-j4bwj"] Jul 20 16:48:59.814120 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:48:59.814080 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="308ef8a5-3aaf-42c5-b2b3-56a044947a8e" path="/var/lib/kubelet/pods/308ef8a5-3aaf-42c5-b2b3-56a044947a8e/volumes" Jul 20 16:49:05.956626 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:05.956586 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw"] Jul 20 16:49:05.957051 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:05.956929 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="308ef8a5-3aaf-42c5-b2b3-56a044947a8e" containerName="console" Jul 20 16:49:05.957051 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:05.956941 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="308ef8a5-3aaf-42c5-b2b3-56a044947a8e" containerName="console" Jul 20 16:49:05.957051 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:05.956999 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="308ef8a5-3aaf-42c5-b2b3-56a044947a8e" containerName="console" Jul 20 16:49:05.960263 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:05.960247 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" Jul 20 16:49:05.964958 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:05.964939 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d8d2p\"" Jul 20 16:49:05.965129 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:05.964939 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 16:49:05.965542 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:05.965519 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 16:49:05.977742 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:05.977719 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw"] Jul 20 16:49:06.027799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:06.027735 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a51a9951-65c5-4708-9255-d42c19ba154d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw\" (UID: \"a51a9951-65c5-4708-9255-d42c19ba154d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" Jul 20 16:49:06.027986 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:06.027803 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjtwn\" (UniqueName: \"kubernetes.io/projected/a51a9951-65c5-4708-9255-d42c19ba154d-kube-api-access-kjtwn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw\" (UID: \"a51a9951-65c5-4708-9255-d42c19ba154d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" Jul 20 16:49:06.027986 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:06.027877 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a51a9951-65c5-4708-9255-d42c19ba154d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw\" (UID: \"a51a9951-65c5-4708-9255-d42c19ba154d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" Jul 20 16:49:06.128575 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:06.128536 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a51a9951-65c5-4708-9255-d42c19ba154d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw\" (UID: \"a51a9951-65c5-4708-9255-d42c19ba154d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" Jul 20 16:49:06.128806 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:06.128598 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a51a9951-65c5-4708-9255-d42c19ba154d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw\" (UID: \"a51a9951-65c5-4708-9255-d42c19ba154d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" Jul 20 16:49:06.128806 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:06.128627 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjtwn\" (UniqueName: \"kubernetes.io/projected/a51a9951-65c5-4708-9255-d42c19ba154d-kube-api-access-kjtwn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw\" (UID: \"a51a9951-65c5-4708-9255-d42c19ba154d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" Jul 20 16:49:06.129057 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:06.129032 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a51a9951-65c5-4708-9255-d42c19ba154d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw\" (UID: \"a51a9951-65c5-4708-9255-d42c19ba154d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" Jul 20 16:49:06.129125 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:06.129063 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a51a9951-65c5-4708-9255-d42c19ba154d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw\" (UID: \"a51a9951-65c5-4708-9255-d42c19ba154d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" Jul 20 16:49:06.142348 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:06.142317 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjtwn\" (UniqueName: \"kubernetes.io/projected/a51a9951-65c5-4708-9255-d42c19ba154d-kube-api-access-kjtwn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw\" (UID: \"a51a9951-65c5-4708-9255-d42c19ba154d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" Jul 20 16:49:06.269078 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:06.268975 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" Jul 20 16:49:06.404408 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:06.404383 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw"] Jul 20 16:49:06.406822 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:49:06.406789 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda51a9951_65c5_4708_9255_d42c19ba154d.slice/crio-b0c5f0e90426fa1385918b7a254e8077cb9711401b8e24e7ed8d8dd820e7aa19 WatchSource:0}: Error finding container b0c5f0e90426fa1385918b7a254e8077cb9711401b8e24e7ed8d8dd820e7aa19: Status 404 returned error can't find the container with id b0c5f0e90426fa1385918b7a254e8077cb9711401b8e24e7ed8d8dd820e7aa19 Jul 20 16:49:06.811681 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:06.811647 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" event={"ID":"a51a9951-65c5-4708-9255-d42c19ba154d","Type":"ContainerStarted","Data":"b0c5f0e90426fa1385918b7a254e8077cb9711401b8e24e7ed8d8dd820e7aa19"} Jul 20 16:49:13.842667 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:13.842628 2566 generic.go:356] "Generic (PLEG): container finished" podID="a51a9951-65c5-4708-9255-d42c19ba154d" containerID="5e28691c183e73f6d8dd839551378c1a5ded6874e8df9e41af9488d69370493f" exitCode=0 Jul 20 16:49:13.843052 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:13.842697 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" event={"ID":"a51a9951-65c5-4708-9255-d42c19ba154d","Type":"ContainerDied","Data":"5e28691c183e73f6d8dd839551378c1a5ded6874e8df9e41af9488d69370493f"} Jul 20 16:49:16.857029 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:16.856995 2566 generic.go:356] "Generic (PLEG): container finished" podID="a51a9951-65c5-4708-9255-d42c19ba154d" containerID="7a2c5a74e8ea4e8a612680b0ce209d472f521876b1d1fc569debd5f8004e1413" exitCode=0 Jul 20 16:49:16.857389 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:16.857071 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" event={"ID":"a51a9951-65c5-4708-9255-d42c19ba154d","Type":"ContainerDied","Data":"7a2c5a74e8ea4e8a612680b0ce209d472f521876b1d1fc569debd5f8004e1413"} Jul 20 16:49:24.896268 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:24.896235 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" event={"ID":"a51a9951-65c5-4708-9255-d42c19ba154d","Type":"ContainerStarted","Data":"7668bfd87e222aca609fb07e387c1aea8ed50e796c432c10a58b63a524d09c65"} Jul 20 16:49:24.924141 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:24.924097 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" podStartSLOduration=2.35379745 podStartE2EDuration="19.924079993s" totalImagesPullingTime="17.570282543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:49:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:49:24.922584461 +0000 UTC m=+229.708583542" watchObservedRunningTime="2026-07-20 16:49:24.924079993 +0000 UTC m=+229.710079071" Jul 20 16:49:25.902328 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:25.902287 2566 generic.go:356] "Generic (PLEG): container finished" podID="a51a9951-65c5-4708-9255-d42c19ba154d" containerID="7668bfd87e222aca609fb07e387c1aea8ed50e796c432c10a58b63a524d09c65" exitCode=0 Jul 20 16:49:25.902707 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:25.902411 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" event={"ID":"a51a9951-65c5-4708-9255-d42c19ba154d","Type":"ContainerDied","Data":"7668bfd87e222aca609fb07e387c1aea8ed50e796c432c10a58b63a524d09c65"} Jul 20 16:49:27.052631 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:27.052593 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" Jul 20 16:49:27.119463 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:27.119423 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjtwn\" (UniqueName: \"kubernetes.io/projected/a51a9951-65c5-4708-9255-d42c19ba154d-kube-api-access-kjtwn\") pod \"a51a9951-65c5-4708-9255-d42c19ba154d\" (UID: \"a51a9951-65c5-4708-9255-d42c19ba154d\") " Jul 20 16:49:27.119661 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:27.119512 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a51a9951-65c5-4708-9255-d42c19ba154d-util\") pod \"a51a9951-65c5-4708-9255-d42c19ba154d\" (UID: \"a51a9951-65c5-4708-9255-d42c19ba154d\") " Jul 20 16:49:27.119661 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:27.119556 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a51a9951-65c5-4708-9255-d42c19ba154d-bundle\") pod \"a51a9951-65c5-4708-9255-d42c19ba154d\" (UID: \"a51a9951-65c5-4708-9255-d42c19ba154d\") " Jul 20 16:49:27.120239 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:27.120213 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51a9951-65c5-4708-9255-d42c19ba154d-bundle" (OuterVolumeSpecName: "bundle") pod "a51a9951-65c5-4708-9255-d42c19ba154d" (UID: "a51a9951-65c5-4708-9255-d42c19ba154d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:49:27.121858 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:27.121836 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a51a9951-65c5-4708-9255-d42c19ba154d-kube-api-access-kjtwn" (OuterVolumeSpecName: "kube-api-access-kjtwn") pod "a51a9951-65c5-4708-9255-d42c19ba154d" (UID: "a51a9951-65c5-4708-9255-d42c19ba154d"). InnerVolumeSpecName "kube-api-access-kjtwn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:49:27.123547 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:27.123521 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51a9951-65c5-4708-9255-d42c19ba154d-util" (OuterVolumeSpecName: "util") pod "a51a9951-65c5-4708-9255-d42c19ba154d" (UID: "a51a9951-65c5-4708-9255-d42c19ba154d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:49:27.220252 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:27.220173 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a51a9951-65c5-4708-9255-d42c19ba154d-util\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:49:27.220252 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:27.220200 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a51a9951-65c5-4708-9255-d42c19ba154d-bundle\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:49:27.220252 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:27.220209 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjtwn\" (UniqueName: \"kubernetes.io/projected/a51a9951-65c5-4708-9255-d42c19ba154d-kube-api-access-kjtwn\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:49:27.912265 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:27.912236 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" Jul 20 16:49:27.912396 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:27.912227 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l7lbw" event={"ID":"a51a9951-65c5-4708-9255-d42c19ba154d","Type":"ContainerDied","Data":"b0c5f0e90426fa1385918b7a254e8077cb9711401b8e24e7ed8d8dd820e7aa19"} Jul 20 16:49:27.912396 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:27.912382 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0c5f0e90426fa1385918b7a254e8077cb9711401b8e24e7ed8d8dd820e7aa19" Jul 20 16:49:34.129436 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.129400 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vpf56"] Jul 20 16:49:34.129903 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.129714 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a51a9951-65c5-4708-9255-d42c19ba154d" containerName="extract" Jul 20 16:49:34.129903 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.129725 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a51a9951-65c5-4708-9255-d42c19ba154d" containerName="extract" Jul 20 16:49:34.129903 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.129741 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a51a9951-65c5-4708-9255-d42c19ba154d" containerName="pull" Jul 20 16:49:34.129903 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.129746 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a51a9951-65c5-4708-9255-d42c19ba154d" containerName="pull" Jul 20 16:49:34.129903 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.129762 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a51a9951-65c5-4708-9255-d42c19ba154d" containerName="util" Jul 20 16:49:34.129903 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.129768 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a51a9951-65c5-4708-9255-d42c19ba154d" containerName="util" Jul 20 16:49:34.129903 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.129832 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a51a9951-65c5-4708-9255-d42c19ba154d" containerName="extract" Jul 20 16:49:34.133433 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.133412 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vpf56" Jul 20 16:49:34.135593 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.135570 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-c88qf\"" Jul 20 16:49:34.136215 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.136198 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 20 16:49:34.136312 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.136226 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 20 16:49:34.149969 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.149944 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vpf56"] Jul 20 16:49:34.180704 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.180673 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dda5469f-cab1-4a5f-9974-33f818778df0-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vpf56\" (UID: \"dda5469f-cab1-4a5f-9974-33f818778df0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vpf56" Jul 20 16:49:34.180912 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.180728 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pld8\" (UniqueName: \"kubernetes.io/projected/dda5469f-cab1-4a5f-9974-33f818778df0-kube-api-access-4pld8\") pod \"cert-manager-operator-controller-manager-55c444bf86-vpf56\" (UID: \"dda5469f-cab1-4a5f-9974-33f818778df0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vpf56" Jul 20 16:49:34.281718 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.281681 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pld8\" (UniqueName: \"kubernetes.io/projected/dda5469f-cab1-4a5f-9974-33f818778df0-kube-api-access-4pld8\") pod \"cert-manager-operator-controller-manager-55c444bf86-vpf56\" (UID: \"dda5469f-cab1-4a5f-9974-33f818778df0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vpf56" Jul 20 16:49:34.281910 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.281810 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dda5469f-cab1-4a5f-9974-33f818778df0-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vpf56\" (UID: \"dda5469f-cab1-4a5f-9974-33f818778df0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vpf56" Jul 20 16:49:34.282211 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.282193 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dda5469f-cab1-4a5f-9974-33f818778df0-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vpf56\" (UID: \"dda5469f-cab1-4a5f-9974-33f818778df0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vpf56" Jul 20 16:49:34.294861 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.294830 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pld8\" (UniqueName: \"kubernetes.io/projected/dda5469f-cab1-4a5f-9974-33f818778df0-kube-api-access-4pld8\") pod \"cert-manager-operator-controller-manager-55c444bf86-vpf56\" (UID: \"dda5469f-cab1-4a5f-9974-33f818778df0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vpf56" Jul 20 16:49:34.442239 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.442153 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vpf56" Jul 20 16:49:34.581474 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.581449 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vpf56"] Jul 20 16:49:34.584410 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:49:34.584380 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda5469f_cab1_4a5f_9974_33f818778df0.slice/crio-f77a68c691046074e07f2c59bf18a1ff9c634bf7769bb6119b7d10155d7147bf WatchSource:0}: Error finding container f77a68c691046074e07f2c59bf18a1ff9c634bf7769bb6119b7d10155d7147bf: Status 404 returned error can't find the container with id f77a68c691046074e07f2c59bf18a1ff9c634bf7769bb6119b7d10155d7147bf Jul 20 16:49:34.941349 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:34.941315 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vpf56" event={"ID":"dda5469f-cab1-4a5f-9974-33f818778df0","Type":"ContainerStarted","Data":"f77a68c691046074e07f2c59bf18a1ff9c634bf7769bb6119b7d10155d7147bf"} Jul 20 16:49:36.951958 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:36.951916 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vpf56" event={"ID":"dda5469f-cab1-4a5f-9974-33f818778df0","Type":"ContainerStarted","Data":"1ff5c08f5df3c0b4b5d4607e74afc27fba37208b437f1c512983c11d08dd4375"} Jul 20 16:49:36.975699 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:36.975633 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vpf56" podStartSLOduration=1.2755998929999999 podStartE2EDuration="2.975617022s" totalImagesPullingTime="1.700017129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:49:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:49:36.975537962 +0000 UTC m=+241.761537041" watchObservedRunningTime="2026-07-20 16:49:36.975617022 +0000 UTC m=+241.761616101" Jul 20 16:49:38.050219 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.050179 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv"] Jul 20 16:49:38.052866 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.052845 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv" Jul 20 16:49:38.057849 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.057830 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d8d2p\"" Jul 20 16:49:38.058006 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.057987 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 16:49:38.058195 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.058167 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 16:49:38.074338 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.074313 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv"] Jul 20 16:49:38.113962 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.113926 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5f578b2-0678-4769-a168-668e3dee3d43-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv\" (UID: \"e5f578b2-0678-4769-a168-668e3dee3d43\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv" Jul 20 16:49:38.114138 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.113977 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9rbr\" (UniqueName: \"kubernetes.io/projected/e5f578b2-0678-4769-a168-668e3dee3d43-kube-api-access-t9rbr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv\" (UID: \"e5f578b2-0678-4769-a168-668e3dee3d43\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv" Jul 20 16:49:38.114138 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.113999 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5f578b2-0678-4769-a168-668e3dee3d43-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv\" (UID: \"e5f578b2-0678-4769-a168-668e3dee3d43\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv" Jul 20 16:49:38.215369 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.215339 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5f578b2-0678-4769-a168-668e3dee3d43-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv\" (UID: \"e5f578b2-0678-4769-a168-668e3dee3d43\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv" Jul 20 16:49:38.215555 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.215395 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9rbr\" (UniqueName: \"kubernetes.io/projected/e5f578b2-0678-4769-a168-668e3dee3d43-kube-api-access-t9rbr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv\" (UID: \"e5f578b2-0678-4769-a168-668e3dee3d43\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv" Jul 20 16:49:38.215555 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.215420 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5f578b2-0678-4769-a168-668e3dee3d43-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv\" (UID: \"e5f578b2-0678-4769-a168-668e3dee3d43\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv" Jul 20 16:49:38.215834 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.215812 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5f578b2-0678-4769-a168-668e3dee3d43-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv\" (UID: \"e5f578b2-0678-4769-a168-668e3dee3d43\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv" Jul 20 16:49:38.215899 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.215849 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5f578b2-0678-4769-a168-668e3dee3d43-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv\" (UID: \"e5f578b2-0678-4769-a168-668e3dee3d43\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv" Jul 20 16:49:38.233012 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.232988 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9rbr\" (UniqueName: \"kubernetes.io/projected/e5f578b2-0678-4769-a168-668e3dee3d43-kube-api-access-t9rbr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv\" (UID: \"e5f578b2-0678-4769-a168-668e3dee3d43\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv" Jul 20 16:49:38.362184 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.362152 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv" Jul 20 16:49:38.484853 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.484819 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-m47r4"] Jul 20 16:49:38.487621 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.487599 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-m47r4" Jul 20 16:49:38.490548 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.490528 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 20 16:49:38.490646 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.490586 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 20 16:49:38.517956 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.517885 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74efb3af-6c15-4fff-b498-4c5eb40f37c9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-m47r4\" (UID: \"74efb3af-6c15-4fff-b498-4c5eb40f37c9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-m47r4" Jul 20 16:49:38.518121 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.518024 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4ccd\" (UniqueName: \"kubernetes.io/projected/74efb3af-6c15-4fff-b498-4c5eb40f37c9-kube-api-access-q4ccd\") pod \"cert-manager-webhook-6486ffd997-m47r4\" (UID: \"74efb3af-6c15-4fff-b498-4c5eb40f37c9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-m47r4" Jul 20 16:49:38.518281 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.518191 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-m47r4"] Jul 20 16:49:38.520583 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.520559 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv"] Jul 20 16:49:38.521693 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:49:38.521669 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5f578b2_0678_4769_a168_668e3dee3d43.slice/crio-3662db9f3e636545d4e0d7f38f6bc48e6e55ec67f7d9387abe676a7434bb49ca WatchSource:0}: Error finding container 3662db9f3e636545d4e0d7f38f6bc48e6e55ec67f7d9387abe676a7434bb49ca: Status 404 returned error can't find the container with id 3662db9f3e636545d4e0d7f38f6bc48e6e55ec67f7d9387abe676a7434bb49ca Jul 20 16:49:38.618921 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.618842 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4ccd\" (UniqueName: \"kubernetes.io/projected/74efb3af-6c15-4fff-b498-4c5eb40f37c9-kube-api-access-q4ccd\") pod \"cert-manager-webhook-6486ffd997-m47r4\" (UID: \"74efb3af-6c15-4fff-b498-4c5eb40f37c9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-m47r4" Jul 20 16:49:38.618921 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.618887 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74efb3af-6c15-4fff-b498-4c5eb40f37c9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-m47r4\" (UID: \"74efb3af-6c15-4fff-b498-4c5eb40f37c9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-m47r4" Jul 20 16:49:38.637291 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.637268 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74efb3af-6c15-4fff-b498-4c5eb40f37c9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-m47r4\" (UID: \"74efb3af-6c15-4fff-b498-4c5eb40f37c9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-m47r4" Jul 20 16:49:38.638635 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.638614 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4ccd\" (UniqueName: \"kubernetes.io/projected/74efb3af-6c15-4fff-b498-4c5eb40f37c9-kube-api-access-q4ccd\") pod \"cert-manager-webhook-6486ffd997-m47r4\" (UID: \"74efb3af-6c15-4fff-b498-4c5eb40f37c9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-m47r4" Jul 20 16:49:38.812533 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.812503 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-m47r4" Jul 20 16:49:38.941835 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.941813 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-m47r4"] Jul 20 16:49:38.944114 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:49:38.944086 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74efb3af_6c15_4fff_b498_4c5eb40f37c9.slice/crio-c92122f4c066adbf9a0a6682a5d56ac9eaa3a38c84faa4cafc2da2ec932521e1 WatchSource:0}: Error finding container c92122f4c066adbf9a0a6682a5d56ac9eaa3a38c84faa4cafc2da2ec932521e1: Status 404 returned error can't find the container with id c92122f4c066adbf9a0a6682a5d56ac9eaa3a38c84faa4cafc2da2ec932521e1 Jul 20 16:49:38.961133 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.961103 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-m47r4" event={"ID":"74efb3af-6c15-4fff-b498-4c5eb40f37c9","Type":"ContainerStarted","Data":"c92122f4c066adbf9a0a6682a5d56ac9eaa3a38c84faa4cafc2da2ec932521e1"} Jul 20 16:49:38.962539 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.962517 2566 generic.go:356] "Generic (PLEG): container finished" podID="e5f578b2-0678-4769-a168-668e3dee3d43" containerID="0adfa25e355e23f0902cb846808b1f45db59240090eb62bc75928fa70f0132f8" exitCode=0 Jul 20 16:49:38.962630 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.962605 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv" event={"ID":"e5f578b2-0678-4769-a168-668e3dee3d43","Type":"ContainerDied","Data":"0adfa25e355e23f0902cb846808b1f45db59240090eb62bc75928fa70f0132f8"} Jul 20 16:49:38.962677 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:38.962657 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv" event={"ID":"e5f578b2-0678-4769-a168-668e3dee3d43","Type":"ContainerStarted","Data":"3662db9f3e636545d4e0d7f38f6bc48e6e55ec67f7d9387abe676a7434bb49ca"} Jul 20 16:49:40.936723 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:40.936688 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-tm5w6"] Jul 20 16:49:40.940491 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:40.940467 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tm5w6" Jul 20 16:49:40.942948 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:40.942916 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-hpws9\"" Jul 20 16:49:40.951418 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:40.951390 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-tm5w6"] Jul 20 16:49:41.041876 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:41.041837 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c036bf5-a1fb-4462-84fb-ae6d2040ad7c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-tm5w6\" (UID: \"1c036bf5-a1fb-4462-84fb-ae6d2040ad7c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tm5w6" Jul 20 16:49:41.042047 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:41.041909 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcfzk\" (UniqueName: \"kubernetes.io/projected/1c036bf5-a1fb-4462-84fb-ae6d2040ad7c-kube-api-access-hcfzk\") pod \"cert-manager-cainjector-559cdd8c87-tm5w6\" (UID: \"1c036bf5-a1fb-4462-84fb-ae6d2040ad7c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tm5w6" Jul 20 16:49:41.143327 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:41.143021 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c036bf5-a1fb-4462-84fb-ae6d2040ad7c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-tm5w6\" (UID: \"1c036bf5-a1fb-4462-84fb-ae6d2040ad7c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tm5w6" Jul 20 16:49:41.143327 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:41.143098 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcfzk\" (UniqueName: \"kubernetes.io/projected/1c036bf5-a1fb-4462-84fb-ae6d2040ad7c-kube-api-access-hcfzk\") pod \"cert-manager-cainjector-559cdd8c87-tm5w6\" (UID: \"1c036bf5-a1fb-4462-84fb-ae6d2040ad7c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tm5w6" Jul 20 16:49:41.152458 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:41.152424 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c036bf5-a1fb-4462-84fb-ae6d2040ad7c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-tm5w6\" (UID: \"1c036bf5-a1fb-4462-84fb-ae6d2040ad7c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tm5w6" Jul 20 16:49:41.152458 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:41.152445 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcfzk\" (UniqueName: \"kubernetes.io/projected/1c036bf5-a1fb-4462-84fb-ae6d2040ad7c-kube-api-access-hcfzk\") pod \"cert-manager-cainjector-559cdd8c87-tm5w6\" (UID: \"1c036bf5-a1fb-4462-84fb-ae6d2040ad7c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tm5w6" Jul 20 16:49:41.254607 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:41.254532 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tm5w6" Jul 20 16:49:42.730040 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:42.729991 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-tm5w6"] Jul 20 16:49:42.735488 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:49:42.735464 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c036bf5_a1fb_4462_84fb_ae6d2040ad7c.slice/crio-e6bca7172222e5265e90f9ad96868a5a67e8c3ca55c2e619badfe0bf4967965f WatchSource:0}: Error finding container e6bca7172222e5265e90f9ad96868a5a67e8c3ca55c2e619badfe0bf4967965f: Status 404 returned error can't find the container with id e6bca7172222e5265e90f9ad96868a5a67e8c3ca55c2e619badfe0bf4967965f Jul 20 16:49:42.984146 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:42.984046 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tm5w6" event={"ID":"1c036bf5-a1fb-4462-84fb-ae6d2040ad7c","Type":"ContainerStarted","Data":"79322114f61d3354908f2c13fb1c513a232abe6cef183c3587ff26ededeb83bc"} Jul 20 16:49:42.984146 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:42.984102 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tm5w6" event={"ID":"1c036bf5-a1fb-4462-84fb-ae6d2040ad7c","Type":"ContainerStarted","Data":"e6bca7172222e5265e90f9ad96868a5a67e8c3ca55c2e619badfe0bf4967965f"} Jul 20 16:49:42.985662 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:42.985632 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-m47r4" event={"ID":"74efb3af-6c15-4fff-b498-4c5eb40f37c9","Type":"ContainerStarted","Data":"afdd4ecbcf31318e376fa38924c81b4c62ade6920b0376b5881f4e1752630387"} Jul 20 16:49:42.985804 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:42.985719 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-m47r4" Jul 20 16:49:42.987699 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:42.987677 2566 generic.go:356] "Generic (PLEG): container finished" podID="e5f578b2-0678-4769-a168-668e3dee3d43" containerID="5ee9df71de0d37f5a56d074e77bbdefb9edca93352d28b3de8210a5cfcdb513c" exitCode=0 Jul 20 16:49:42.987801 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:42.987712 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv" event={"ID":"e5f578b2-0678-4769-a168-668e3dee3d43","Type":"ContainerDied","Data":"5ee9df71de0d37f5a56d074e77bbdefb9edca93352d28b3de8210a5cfcdb513c"} Jul 20 16:49:42.999971 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:42.999925 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tm5w6" podStartSLOduration=2.999906428 podStartE2EDuration="2.999906428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:49:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:49:42.999156805 +0000 UTC m=+247.785155884" watchObservedRunningTime="2026-07-20 16:49:42.999906428 +0000 UTC m=+247.785905509" Jul 20 16:49:43.038468 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:43.038419 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-m47r4" podStartSLOduration=1.385226103 podStartE2EDuration="5.038399011s" totalImagesPullingTime="3.653172908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:49:43.036091843 +0000 UTC m=+247.822090918" watchObservedRunningTime="2026-07-20 16:49:43.038399011 +0000 UTC m=+247.824398090" Jul 20 16:49:43.999408 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:43.999323 2566 generic.go:356] "Generic (PLEG): container finished" podID="e5f578b2-0678-4769-a168-668e3dee3d43" containerID="d418e73b866d85d76ee27e8718a86f5b5f25c6302232033944be410c1ca6000f" exitCode=0 Jul 20 16:49:43.999800 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:43.999414 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv" event={"ID":"e5f578b2-0678-4769-a168-668e3dee3d43","Type":"ContainerDied","Data":"d418e73b866d85d76ee27e8718a86f5b5f25c6302232033944be410c1ca6000f"} Jul 20 16:49:45.126896 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:45.126875 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv" Jul 20 16:49:45.179583 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:45.179542 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5f578b2-0678-4769-a168-668e3dee3d43-util\") pod \"e5f578b2-0678-4769-a168-668e3dee3d43\" (UID: \"e5f578b2-0678-4769-a168-668e3dee3d43\") " Jul 20 16:49:45.179805 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:45.179599 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9rbr\" (UniqueName: \"kubernetes.io/projected/e5f578b2-0678-4769-a168-668e3dee3d43-kube-api-access-t9rbr\") pod \"e5f578b2-0678-4769-a168-668e3dee3d43\" (UID: \"e5f578b2-0678-4769-a168-668e3dee3d43\") " Jul 20 16:49:45.179805 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:45.179622 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5f578b2-0678-4769-a168-668e3dee3d43-bundle\") pod \"e5f578b2-0678-4769-a168-668e3dee3d43\" (UID: \"e5f578b2-0678-4769-a168-668e3dee3d43\") " Jul 20 16:49:45.180048 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:45.180022 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5f578b2-0678-4769-a168-668e3dee3d43-bundle" (OuterVolumeSpecName: "bundle") pod "e5f578b2-0678-4769-a168-668e3dee3d43" (UID: "e5f578b2-0678-4769-a168-668e3dee3d43"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:49:45.181959 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:45.181935 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5f578b2-0678-4769-a168-668e3dee3d43-kube-api-access-t9rbr" (OuterVolumeSpecName: "kube-api-access-t9rbr") pod "e5f578b2-0678-4769-a168-668e3dee3d43" (UID: "e5f578b2-0678-4769-a168-668e3dee3d43"). InnerVolumeSpecName "kube-api-access-t9rbr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:49:45.184583 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:45.184556 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5f578b2-0678-4769-a168-668e3dee3d43-util" (OuterVolumeSpecName: "util") pod "e5f578b2-0678-4769-a168-668e3dee3d43" (UID: "e5f578b2-0678-4769-a168-668e3dee3d43"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:49:45.280445 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:45.280364 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5f578b2-0678-4769-a168-668e3dee3d43-util\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:49:45.280445 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:45.280390 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9rbr\" (UniqueName: \"kubernetes.io/projected/e5f578b2-0678-4769-a168-668e3dee3d43-kube-api-access-t9rbr\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:49:45.280445 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:45.280401 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5f578b2-0678-4769-a168-668e3dee3d43-bundle\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:49:46.009721 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:46.009692 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv" Jul 20 16:49:46.009911 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:46.009694 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvk9lv" event={"ID":"e5f578b2-0678-4769-a168-668e3dee3d43","Type":"ContainerDied","Data":"3662db9f3e636545d4e0d7f38f6bc48e6e55ec67f7d9387abe676a7434bb49ca"} Jul 20 16:49:46.009911 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:46.009827 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3662db9f3e636545d4e0d7f38f6bc48e6e55ec67f7d9387abe676a7434bb49ca" Jul 20 16:49:49.001926 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:49.001898 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-m47r4" Jul 20 16:49:50.489945 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.489909 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c5tst"] Jul 20 16:49:50.490317 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.490239 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5f578b2-0678-4769-a168-668e3dee3d43" containerName="pull" Jul 20 16:49:50.490317 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.490250 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5f578b2-0678-4769-a168-668e3dee3d43" containerName="pull" Jul 20 16:49:50.490317 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.490260 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5f578b2-0678-4769-a168-668e3dee3d43" containerName="extract" Jul 20 16:49:50.490317 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.490266 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5f578b2-0678-4769-a168-668e3dee3d43" containerName="extract" Jul 20 16:49:50.490317 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.490280 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5f578b2-0678-4769-a168-668e3dee3d43" containerName="util" Jul 20 16:49:50.490317 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.490285 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5f578b2-0678-4769-a168-668e3dee3d43" containerName="util" Jul 20 16:49:50.490493 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.490342 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5f578b2-0678-4769-a168-668e3dee3d43" containerName="extract" Jul 20 16:49:50.492617 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.492602 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-c5tst" Jul 20 16:49:50.495337 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.495317 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-bf2th\"" Jul 20 16:49:50.511000 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.510969 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c5tst"] Jul 20 16:49:50.623648 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.623615 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsrzk\" (UniqueName: \"kubernetes.io/projected/950845a4-5331-4e3b-a342-faf5526be2a4-kube-api-access-lsrzk\") pod \"cert-manager-55b4d954d5-c5tst\" (UID: \"950845a4-5331-4e3b-a342-faf5526be2a4\") " pod="cert-manager/cert-manager-55b4d954d5-c5tst" Jul 20 16:49:50.623840 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.623674 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/950845a4-5331-4e3b-a342-faf5526be2a4-bound-sa-token\") pod \"cert-manager-55b4d954d5-c5tst\" (UID: \"950845a4-5331-4e3b-a342-faf5526be2a4\") " pod="cert-manager/cert-manager-55b4d954d5-c5tst" Jul 20 16:49:50.724444 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.724405 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsrzk\" (UniqueName: \"kubernetes.io/projected/950845a4-5331-4e3b-a342-faf5526be2a4-kube-api-access-lsrzk\") pod \"cert-manager-55b4d954d5-c5tst\" (UID: \"950845a4-5331-4e3b-a342-faf5526be2a4\") " pod="cert-manager/cert-manager-55b4d954d5-c5tst" Jul 20 16:49:50.724641 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.724461 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/950845a4-5331-4e3b-a342-faf5526be2a4-bound-sa-token\") pod \"cert-manager-55b4d954d5-c5tst\" (UID: \"950845a4-5331-4e3b-a342-faf5526be2a4\") " pod="cert-manager/cert-manager-55b4d954d5-c5tst" Jul 20 16:49:50.740372 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.740299 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsrzk\" (UniqueName: \"kubernetes.io/projected/950845a4-5331-4e3b-a342-faf5526be2a4-kube-api-access-lsrzk\") pod \"cert-manager-55b4d954d5-c5tst\" (UID: \"950845a4-5331-4e3b-a342-faf5526be2a4\") " pod="cert-manager/cert-manager-55b4d954d5-c5tst" Jul 20 16:49:50.740729 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.740708 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/950845a4-5331-4e3b-a342-faf5526be2a4-bound-sa-token\") pod \"cert-manager-55b4d954d5-c5tst\" (UID: \"950845a4-5331-4e3b-a342-faf5526be2a4\") " pod="cert-manager/cert-manager-55b4d954d5-c5tst" Jul 20 16:49:50.801513 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.801482 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-c5tst" Jul 20 16:49:50.943094 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:50.942921 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c5tst"] Jul 20 16:49:50.945787 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:49:50.945740 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod950845a4_5331_4e3b_a342_faf5526be2a4.slice/crio-90abc851444e572601338b118a4aa6dbbbb22542618378ed33eb2d2dc7adc623 WatchSource:0}: Error finding container 90abc851444e572601338b118a4aa6dbbbb22542618378ed33eb2d2dc7adc623: Status 404 returned error can't find the container with id 90abc851444e572601338b118a4aa6dbbbb22542618378ed33eb2d2dc7adc623 Jul 20 16:49:51.035458 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:51.035412 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-c5tst" event={"ID":"950845a4-5331-4e3b-a342-faf5526be2a4","Type":"ContainerStarted","Data":"9ae0e2d5413565ef99965b037061ebc17c8ddfc701892df23605ea37db2c9cb9"} Jul 20 16:49:51.035636 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:51.035473 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-c5tst" event={"ID":"950845a4-5331-4e3b-a342-faf5526be2a4","Type":"ContainerStarted","Data":"90abc851444e572601338b118a4aa6dbbbb22542618378ed33eb2d2dc7adc623"} Jul 20 16:49:51.060377 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:51.060333 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-c5tst" podStartSLOduration=1.060316566 podStartE2EDuration="1.060316566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:49:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:49:51.059883448 +0000 UTC m=+255.845882526" watchObservedRunningTime="2026-07-20 16:49:51.060316566 +0000 UTC m=+255.846315644" Jul 20 16:49:56.184796 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:56.184705 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pz67k"] Jul 20 16:49:56.186882 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:56.186864 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pz67k" Jul 20 16:49:56.190019 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:56.189998 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-mlnzb\"" Jul 20 16:49:56.190645 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:56.190629 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 20 16:49:56.190685 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:56.190646 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 20 16:49:56.204033 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:56.203998 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pz67k"] Jul 20 16:49:56.270829 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:56.270759 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdrs5\" (UniqueName: \"kubernetes.io/projected/1a756eb7-f252-4777-806b-ec4162928819-kube-api-access-cdrs5\") pod \"openshift-lws-operator-fd8ccff4c-pz67k\" (UID: \"1a756eb7-f252-4777-806b-ec4162928819\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pz67k" Jul 20 16:49:56.270829 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:56.270832 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a756eb7-f252-4777-806b-ec4162928819-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pz67k\" (UID: \"1a756eb7-f252-4777-806b-ec4162928819\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pz67k" Jul 20 16:49:56.372145 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:56.372083 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdrs5\" (UniqueName: \"kubernetes.io/projected/1a756eb7-f252-4777-806b-ec4162928819-kube-api-access-cdrs5\") pod \"openshift-lws-operator-fd8ccff4c-pz67k\" (UID: \"1a756eb7-f252-4777-806b-ec4162928819\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pz67k" Jul 20 16:49:56.372145 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:56.372148 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a756eb7-f252-4777-806b-ec4162928819-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pz67k\" (UID: \"1a756eb7-f252-4777-806b-ec4162928819\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pz67k" Jul 20 16:49:56.372565 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:56.372542 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a756eb7-f252-4777-806b-ec4162928819-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pz67k\" (UID: \"1a756eb7-f252-4777-806b-ec4162928819\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pz67k" Jul 20 16:49:56.381797 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:56.381748 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdrs5\" (UniqueName: \"kubernetes.io/projected/1a756eb7-f252-4777-806b-ec4162928819-kube-api-access-cdrs5\") pod \"openshift-lws-operator-fd8ccff4c-pz67k\" (UID: \"1a756eb7-f252-4777-806b-ec4162928819\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pz67k" Jul 20 16:49:56.496222 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:56.496143 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pz67k" Jul 20 16:49:56.632923 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:56.632888 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pz67k"] Jul 20 16:49:56.636852 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:49:56.636819 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a756eb7_f252_4777_806b_ec4162928819.slice/crio-d95eae94b74a22ca532af4dd3a9357889f24504ba924069412de6b195c3c9dad WatchSource:0}: Error finding container d95eae94b74a22ca532af4dd3a9357889f24504ba924069412de6b195c3c9dad: Status 404 returned error can't find the container with id d95eae94b74a22ca532af4dd3a9357889f24504ba924069412de6b195c3c9dad Jul 20 16:49:57.062206 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:49:57.062166 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pz67k" event={"ID":"1a756eb7-f252-4777-806b-ec4162928819","Type":"ContainerStarted","Data":"d95eae94b74a22ca532af4dd3a9357889f24504ba924069412de6b195c3c9dad"} Jul 20 16:50:00.076933 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:00.076894 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pz67k" event={"ID":"1a756eb7-f252-4777-806b-ec4162928819","Type":"ContainerStarted","Data":"f4e960ab6f40ebc93506b27d15665f7d7bacabc73aea8014869a20ca6e10d562"} Jul 20 16:50:00.099193 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:00.099143 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pz67k" podStartSLOduration=1.382308482 podStartE2EDuration="4.09912628s" totalImagesPullingTime="2.716817798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:49:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:50:00.09757867 +0000 UTC m=+264.883577761" watchObservedRunningTime="2026-07-20 16:50:00.09912628 +0000 UTC m=+264.885125359" Jul 20 16:50:03.946985 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:03.946951 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw"] Jul 20 16:50:03.949902 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:03.949884 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw" Jul 20 16:50:03.954406 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:03.954389 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 16:50:03.954494 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:03.954455 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 16:50:03.954855 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:03.954842 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d8d2p\"" Jul 20 16:50:03.970413 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:03.970382 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw"] Jul 20 16:50:04.038238 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:04.038202 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d199eab-fd96-483c-958f-621e976f257d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw\" (UID: \"9d199eab-fd96-483c-958f-621e976f257d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw" Jul 20 16:50:04.038403 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:04.038256 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcdff\" (UniqueName: \"kubernetes.io/projected/9d199eab-fd96-483c-958f-621e976f257d-kube-api-access-lcdff\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw\" (UID: \"9d199eab-fd96-483c-958f-621e976f257d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw" Jul 20 16:50:04.038403 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:04.038337 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d199eab-fd96-483c-958f-621e976f257d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw\" (UID: \"9d199eab-fd96-483c-958f-621e976f257d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw" Jul 20 16:50:04.139138 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:04.139100 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d199eab-fd96-483c-958f-621e976f257d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw\" (UID: \"9d199eab-fd96-483c-958f-621e976f257d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw" Jul 20 16:50:04.139315 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:04.139154 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcdff\" (UniqueName: \"kubernetes.io/projected/9d199eab-fd96-483c-958f-621e976f257d-kube-api-access-lcdff\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw\" (UID: \"9d199eab-fd96-483c-958f-621e976f257d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw" Jul 20 16:50:04.139315 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:04.139180 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d199eab-fd96-483c-958f-621e976f257d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw\" (UID: \"9d199eab-fd96-483c-958f-621e976f257d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw" Jul 20 16:50:04.139523 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:04.139505 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d199eab-fd96-483c-958f-621e976f257d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw\" (UID: \"9d199eab-fd96-483c-958f-621e976f257d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw" Jul 20 16:50:04.139558 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:04.139522 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d199eab-fd96-483c-958f-621e976f257d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw\" (UID: \"9d199eab-fd96-483c-958f-621e976f257d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw" Jul 20 16:50:04.153478 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:04.153443 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcdff\" (UniqueName: \"kubernetes.io/projected/9d199eab-fd96-483c-958f-621e976f257d-kube-api-access-lcdff\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw\" (UID: \"9d199eab-fd96-483c-958f-621e976f257d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw" Jul 20 16:50:04.259014 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:04.258927 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw" Jul 20 16:50:04.396924 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:04.396901 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw"] Jul 20 16:50:04.399029 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:50:04.399001 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d199eab_fd96_483c_958f_621e976f257d.slice/crio-ab85e80fa978be98cba90b6bd8c68a84fc54267f2623ef1d746b6b0b9e652a10 WatchSource:0}: Error finding container ab85e80fa978be98cba90b6bd8c68a84fc54267f2623ef1d746b6b0b9e652a10: Status 404 returned error can't find the container with id ab85e80fa978be98cba90b6bd8c68a84fc54267f2623ef1d746b6b0b9e652a10 Jul 20 16:50:05.100367 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:05.100330 2566 generic.go:356] "Generic (PLEG): container finished" podID="9d199eab-fd96-483c-958f-621e976f257d" containerID="8cb50b885e1a9cc2439e0ff7b6347b860832a09f9f81c0ffd2ca5c5f03862e4b" exitCode=0 Jul 20 16:50:05.100741 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:05.100411 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw" event={"ID":"9d199eab-fd96-483c-958f-621e976f257d","Type":"ContainerDied","Data":"8cb50b885e1a9cc2439e0ff7b6347b860832a09f9f81c0ffd2ca5c5f03862e4b"} Jul 20 16:50:05.100741 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:05.100460 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw" event={"ID":"9d199eab-fd96-483c-958f-621e976f257d","Type":"ContainerStarted","Data":"ab85e80fa978be98cba90b6bd8c68a84fc54267f2623ef1d746b6b0b9e652a10"} Jul 20 16:50:06.106980 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:06.106946 2566 generic.go:356] "Generic (PLEG): container finished" podID="9d199eab-fd96-483c-958f-621e976f257d" containerID="6cc2a7cbd2d9288b640c173c5921e85059d658a97c8a5958d14c15aceca3adbc" exitCode=0 Jul 20 16:50:06.107389 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:06.107009 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw" event={"ID":"9d199eab-fd96-483c-958f-621e976f257d","Type":"ContainerDied","Data":"6cc2a7cbd2d9288b640c173c5921e85059d658a97c8a5958d14c15aceca3adbc"} Jul 20 16:50:07.113543 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:07.113513 2566 generic.go:356] "Generic (PLEG): container finished" podID="9d199eab-fd96-483c-958f-621e976f257d" containerID="9fc4a3d5bdb440a404abb13fd690c9686c5e20c716b12ce478ac6eeb2fe95169" exitCode=0 Jul 20 16:50:07.113951 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:07.113586 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw" event={"ID":"9d199eab-fd96-483c-958f-621e976f257d","Type":"ContainerDied","Data":"9fc4a3d5bdb440a404abb13fd690c9686c5e20c716b12ce478ac6eeb2fe95169"} Jul 20 16:50:08.244540 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:08.244516 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw" Jul 20 16:50:08.376767 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:08.376681 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcdff\" (UniqueName: \"kubernetes.io/projected/9d199eab-fd96-483c-958f-621e976f257d-kube-api-access-lcdff\") pod \"9d199eab-fd96-483c-958f-621e976f257d\" (UID: \"9d199eab-fd96-483c-958f-621e976f257d\") " Jul 20 16:50:08.376969 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:08.376800 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d199eab-fd96-483c-958f-621e976f257d-util\") pod \"9d199eab-fd96-483c-958f-621e976f257d\" (UID: \"9d199eab-fd96-483c-958f-621e976f257d\") " Jul 20 16:50:08.376969 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:08.376838 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d199eab-fd96-483c-958f-621e976f257d-bundle\") pod \"9d199eab-fd96-483c-958f-621e976f257d\" (UID: \"9d199eab-fd96-483c-958f-621e976f257d\") " Jul 20 16:50:08.377498 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:08.377476 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d199eab-fd96-483c-958f-621e976f257d-bundle" (OuterVolumeSpecName: "bundle") pod "9d199eab-fd96-483c-958f-621e976f257d" (UID: "9d199eab-fd96-483c-958f-621e976f257d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:50:08.379052 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:08.379029 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d199eab-fd96-483c-958f-621e976f257d-kube-api-access-lcdff" (OuterVolumeSpecName: "kube-api-access-lcdff") pod "9d199eab-fd96-483c-958f-621e976f257d" (UID: "9d199eab-fd96-483c-958f-621e976f257d"). InnerVolumeSpecName "kube-api-access-lcdff". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:50:08.382216 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:08.382174 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d199eab-fd96-483c-958f-621e976f257d-util" (OuterVolumeSpecName: "util") pod "9d199eab-fd96-483c-958f-621e976f257d" (UID: "9d199eab-fd96-483c-958f-621e976f257d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:50:08.477983 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:08.477944 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d199eab-fd96-483c-958f-621e976f257d-util\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:50:08.478154 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:08.477998 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d199eab-fd96-483c-958f-621e976f257d-bundle\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:50:08.478154 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:08.478008 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcdff\" (UniqueName: \"kubernetes.io/projected/9d199eab-fd96-483c-958f-621e976f257d-kube-api-access-lcdff\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:50:09.125335 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:09.125289 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw" event={"ID":"9d199eab-fd96-483c-958f-621e976f257d","Type":"ContainerDied","Data":"ab85e80fa978be98cba90b6bd8c68a84fc54267f2623ef1d746b6b0b9e652a10"} Jul 20 16:50:09.125509 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:09.125348 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab85e80fa978be98cba90b6bd8c68a84fc54267f2623ef1d746b6b0b9e652a10" Jul 20 16:50:09.125509 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:09.125310 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnvshw" Jul 20 16:50:16.541418 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.541383 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn"] Jul 20 16:50:16.541874 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.541715 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d199eab-fd96-483c-958f-621e976f257d" containerName="pull" Jul 20 16:50:16.541874 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.541725 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d199eab-fd96-483c-958f-621e976f257d" containerName="pull" Jul 20 16:50:16.541874 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.541740 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d199eab-fd96-483c-958f-621e976f257d" containerName="util" Jul 20 16:50:16.541874 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.541746 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d199eab-fd96-483c-958f-621e976f257d" containerName="util" Jul 20 16:50:16.541874 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.541754 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d199eab-fd96-483c-958f-621e976f257d" containerName="extract" Jul 20 16:50:16.541874 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.541761 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d199eab-fd96-483c-958f-621e976f257d" containerName="extract" Jul 20 16:50:16.541874 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.541832 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d199eab-fd96-483c-958f-621e976f257d" containerName="extract" Jul 20 16:50:16.598642 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.598614 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn"] Jul 20 16:50:16.598808 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.598722 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:16.602873 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.602846 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 20 16:50:16.603001 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.602881 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zwkbp\"" Jul 20 16:50:16.603054 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.603025 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 20 16:50:16.603125 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.603103 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 20 16:50:16.603239 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.603176 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 20 16:50:16.750360 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.750319 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b7009ce5-2b66-46ec-8d78-648659de2682-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-z97kn\" (UID: \"b7009ce5-2b66-46ec-8d78-648659de2682\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:16.750360 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.750363 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7009ce5-2b66-46ec-8d78-648659de2682-tmp\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-z97kn\" (UID: \"b7009ce5-2b66-46ec-8d78-648659de2682\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:16.750622 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.750387 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b7009ce5-2b66-46ec-8d78-648659de2682-webhook-cert\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-z97kn\" (UID: \"b7009ce5-2b66-46ec-8d78-648659de2682\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:16.750622 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.750546 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jmvd\" (UniqueName: \"kubernetes.io/projected/b7009ce5-2b66-46ec-8d78-648659de2682-kube-api-access-6jmvd\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-z97kn\" (UID: \"b7009ce5-2b66-46ec-8d78-648659de2682\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:16.750622 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.750596 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b7009ce5-2b66-46ec-8d78-648659de2682-manifests\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-z97kn\" (UID: \"b7009ce5-2b66-46ec-8d78-648659de2682\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:16.851638 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.851605 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b7009ce5-2b66-46ec-8d78-648659de2682-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-z97kn\" (UID: \"b7009ce5-2b66-46ec-8d78-648659de2682\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:16.851835 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.851648 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7009ce5-2b66-46ec-8d78-648659de2682-tmp\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-z97kn\" (UID: \"b7009ce5-2b66-46ec-8d78-648659de2682\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:16.851835 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.851670 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b7009ce5-2b66-46ec-8d78-648659de2682-webhook-cert\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-z97kn\" (UID: \"b7009ce5-2b66-46ec-8d78-648659de2682\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:16.851835 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.851752 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jmvd\" (UniqueName: \"kubernetes.io/projected/b7009ce5-2b66-46ec-8d78-648659de2682-kube-api-access-6jmvd\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-z97kn\" (UID: \"b7009ce5-2b66-46ec-8d78-648659de2682\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:16.851835 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.851808 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b7009ce5-2b66-46ec-8d78-648659de2682-manifests\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-z97kn\" (UID: \"b7009ce5-2b66-46ec-8d78-648659de2682\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:16.852218 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.852196 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b7009ce5-2b66-46ec-8d78-648659de2682-manifests\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-z97kn\" (UID: \"b7009ce5-2b66-46ec-8d78-648659de2682\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:16.854257 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.854230 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7009ce5-2b66-46ec-8d78-648659de2682-tmp\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-z97kn\" (UID: \"b7009ce5-2b66-46ec-8d78-648659de2682\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:16.854355 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.854339 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b7009ce5-2b66-46ec-8d78-648659de2682-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-z97kn\" (UID: \"b7009ce5-2b66-46ec-8d78-648659de2682\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:16.854439 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.854422 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b7009ce5-2b66-46ec-8d78-648659de2682-webhook-cert\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-z97kn\" (UID: \"b7009ce5-2b66-46ec-8d78-648659de2682\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:16.875265 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.875233 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jmvd\" (UniqueName: \"kubernetes.io/projected/b7009ce5-2b66-46ec-8d78-648659de2682-kube-api-access-6jmvd\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-z97kn\" (UID: \"b7009ce5-2b66-46ec-8d78-648659de2682\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:16.908858 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:16.908829 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:17.098888 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:50:17.098843 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7009ce5_2b66_46ec_8d78_648659de2682.slice/crio-f03488c5303bd93f949869d22fe4be75be0dae97de089a75e0357d31148e5df7 WatchSource:0}: Error finding container f03488c5303bd93f949869d22fe4be75be0dae97de089a75e0357d31148e5df7: Status 404 returned error can't find the container with id f03488c5303bd93f949869d22fe4be75be0dae97de089a75e0357d31148e5df7 Jul 20 16:50:17.099697 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:17.099674 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn"] Jul 20 16:50:17.163124 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:17.163084 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" event={"ID":"b7009ce5-2b66-46ec-8d78-648659de2682","Type":"ContainerStarted","Data":"f03488c5303bd93f949869d22fe4be75be0dae97de089a75e0357d31148e5df7"} Jul 20 16:50:20.179337 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:20.179296 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" event={"ID":"b7009ce5-2b66-46ec-8d78-648659de2682","Type":"ContainerStarted","Data":"07679a247ad015ec0f9597b11edc0aeb093673464408e1f2ad6ddc113ab148a9"} Jul 20 16:50:21.185760 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:21.185726 2566 generic.go:356] "Generic (PLEG): container finished" podID="b7009ce5-2b66-46ec-8d78-648659de2682" containerID="07679a247ad015ec0f9597b11edc0aeb093673464408e1f2ad6ddc113ab148a9" exitCode=0 Jul 20 16:50:21.186183 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:21.185817 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" event={"ID":"b7009ce5-2b66-46ec-8d78-648659de2682","Type":"ContainerDied","Data":"07679a247ad015ec0f9597b11edc0aeb093673464408e1f2ad6ddc113ab148a9"} Jul 20 16:50:22.192737 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:22.192701 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" event={"ID":"b7009ce5-2b66-46ec-8d78-648659de2682","Type":"ContainerStarted","Data":"0beb4f6a92e66d8e3b7e9ff8e05fdd4296908ca0257bf6800deb3294c89b0f6e"} Jul 20 16:50:22.193155 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:22.192811 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:22.253126 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:22.253066 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" podStartSLOduration=2.01340373 podStartE2EDuration="6.253043335s" totalImagesPullingTime="3.222698731s" totalInitContainerRuntime="1.016940874s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:50:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:50:22.251098102 +0000 UTC m=+287.037097182" watchObservedRunningTime="2026-07-20 16:50:22.253043335 +0000 UTC m=+287.039042414" Jul 20 16:50:33.201511 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:33.201477 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-z97kn" Jul 20 16:50:35.643079 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:35.643052 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 16:50:35.646552 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:35.646532 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 16:50:35.651419 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:35.651387 2566 kubelet.go:1648] "Image garbage collection succeeded" Jul 20 16:50:36.015105 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.015019 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-86568699db-chx9m"] Jul 20 16:50:36.017412 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.017392 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" Jul 20 16:50:36.027380 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.027339 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 20 16:50:36.031362 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.031336 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 20 16:50:36.031468 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.031337 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-49kdd\"" Jul 20 16:50:36.031997 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.031972 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 20 16:50:36.059875 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.059833 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-86568699db-chx9m"] Jul 20 16:50:36.120889 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.120848 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0744defc-d845-428d-b757-60f6204a9d57-manager-config\") pod \"lws-controller-manager-86568699db-chx9m\" (UID: \"0744defc-d845-428d-b757-60f6204a9d57\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" Jul 20 16:50:36.121122 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.120906 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0744defc-d845-428d-b757-60f6204a9d57-cert\") pod \"lws-controller-manager-86568699db-chx9m\" (UID: \"0744defc-d845-428d-b757-60f6204a9d57\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" Jul 20 16:50:36.121122 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.120988 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kzr4\" (UniqueName: \"kubernetes.io/projected/0744defc-d845-428d-b757-60f6204a9d57-kube-api-access-9kzr4\") pod \"lws-controller-manager-86568699db-chx9m\" (UID: \"0744defc-d845-428d-b757-60f6204a9d57\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" Jul 20 16:50:36.121261 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.121104 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0744defc-d845-428d-b757-60f6204a9d57-metrics-cert\") pod \"lws-controller-manager-86568699db-chx9m\" (UID: \"0744defc-d845-428d-b757-60f6204a9d57\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" Jul 20 16:50:36.221943 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.221906 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0744defc-d845-428d-b757-60f6204a9d57-cert\") pod \"lws-controller-manager-86568699db-chx9m\" (UID: \"0744defc-d845-428d-b757-60f6204a9d57\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" Jul 20 16:50:36.222140 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.221961 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kzr4\" (UniqueName: \"kubernetes.io/projected/0744defc-d845-428d-b757-60f6204a9d57-kube-api-access-9kzr4\") pod \"lws-controller-manager-86568699db-chx9m\" (UID: \"0744defc-d845-428d-b757-60f6204a9d57\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" Jul 20 16:50:36.222140 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.222008 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0744defc-d845-428d-b757-60f6204a9d57-metrics-cert\") pod \"lws-controller-manager-86568699db-chx9m\" (UID: \"0744defc-d845-428d-b757-60f6204a9d57\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" Jul 20 16:50:36.222140 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.222053 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0744defc-d845-428d-b757-60f6204a9d57-manager-config\") pod \"lws-controller-manager-86568699db-chx9m\" (UID: \"0744defc-d845-428d-b757-60f6204a9d57\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" Jul 20 16:50:36.222794 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.222755 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0744defc-d845-428d-b757-60f6204a9d57-manager-config\") pod \"lws-controller-manager-86568699db-chx9m\" (UID: \"0744defc-d845-428d-b757-60f6204a9d57\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" Jul 20 16:50:36.224762 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.224731 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0744defc-d845-428d-b757-60f6204a9d57-metrics-cert\") pod \"lws-controller-manager-86568699db-chx9m\" (UID: \"0744defc-d845-428d-b757-60f6204a9d57\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" Jul 20 16:50:36.224969 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.224945 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0744defc-d845-428d-b757-60f6204a9d57-cert\") pod \"lws-controller-manager-86568699db-chx9m\" (UID: \"0744defc-d845-428d-b757-60f6204a9d57\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" Jul 20 16:50:36.244233 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.244199 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kzr4\" (UniqueName: \"kubernetes.io/projected/0744defc-d845-428d-b757-60f6204a9d57-kube-api-access-9kzr4\") pod \"lws-controller-manager-86568699db-chx9m\" (UID: \"0744defc-d845-428d-b757-60f6204a9d57\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" Jul 20 16:50:36.327038 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.326949 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" Jul 20 16:50:36.603408 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:36.603370 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-86568699db-chx9m"] Jul 20 16:50:36.609164 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:50:36.609136 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0744defc_d845_428d_b757_60f6204a9d57.slice/crio-059dd62fe187e3f287ac99813cc133dcdad81b75ad9f7d331bf625dd022eb9a2 WatchSource:0}: Error finding container 059dd62fe187e3f287ac99813cc133dcdad81b75ad9f7d331bf625dd022eb9a2: Status 404 returned error can't find the container with id 059dd62fe187e3f287ac99813cc133dcdad81b75ad9f7d331bf625dd022eb9a2 Jul 20 16:50:37.102058 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.102011 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-769f9dd78c-k2r9q"] Jul 20 16:50:37.105892 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.105874 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-769f9dd78c-k2r9q" Jul 20 16:50:37.117532 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.117512 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 20 16:50:37.118126 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.118100 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-2qb8r\"" Jul 20 16:50:37.118237 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.118103 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 20 16:50:37.118237 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.118105 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 20 16:50:37.118237 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.118109 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 20 16:50:37.154462 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.154426 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-769f9dd78c-k2r9q"] Jul 20 16:50:37.233424 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.233375 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6748e8be-7893-4fc4-9b26-31ce6f90f03d-tls-certs\") pod \"kube-auth-proxy-769f9dd78c-k2r9q\" (UID: \"6748e8be-7893-4fc4-9b26-31ce6f90f03d\") " pod="openshift-ingress/kube-auth-proxy-769f9dd78c-k2r9q" Jul 20 16:50:37.233609 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.233531 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6748e8be-7893-4fc4-9b26-31ce6f90f03d-tmp\") pod \"kube-auth-proxy-769f9dd78c-k2r9q\" (UID: \"6748e8be-7893-4fc4-9b26-31ce6f90f03d\") " pod="openshift-ingress/kube-auth-proxy-769f9dd78c-k2r9q" Jul 20 16:50:37.233685 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.233633 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbn8s\" (UniqueName: \"kubernetes.io/projected/6748e8be-7893-4fc4-9b26-31ce6f90f03d-kube-api-access-vbn8s\") pod \"kube-auth-proxy-769f9dd78c-k2r9q\" (UID: \"6748e8be-7893-4fc4-9b26-31ce6f90f03d\") " pod="openshift-ingress/kube-auth-proxy-769f9dd78c-k2r9q" Jul 20 16:50:37.269860 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.269818 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" event={"ID":"0744defc-d845-428d-b757-60f6204a9d57","Type":"ContainerStarted","Data":"059dd62fe187e3f287ac99813cc133dcdad81b75ad9f7d331bf625dd022eb9a2"} Jul 20 16:50:37.334162 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.334118 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6748e8be-7893-4fc4-9b26-31ce6f90f03d-tls-certs\") pod \"kube-auth-proxy-769f9dd78c-k2r9q\" (UID: \"6748e8be-7893-4fc4-9b26-31ce6f90f03d\") " pod="openshift-ingress/kube-auth-proxy-769f9dd78c-k2r9q" Jul 20 16:50:37.334356 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.334227 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6748e8be-7893-4fc4-9b26-31ce6f90f03d-tmp\") pod \"kube-auth-proxy-769f9dd78c-k2r9q\" (UID: \"6748e8be-7893-4fc4-9b26-31ce6f90f03d\") " pod="openshift-ingress/kube-auth-proxy-769f9dd78c-k2r9q" Jul 20 16:50:37.334356 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.334269 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbn8s\" (UniqueName: \"kubernetes.io/projected/6748e8be-7893-4fc4-9b26-31ce6f90f03d-kube-api-access-vbn8s\") pod \"kube-auth-proxy-769f9dd78c-k2r9q\" (UID: \"6748e8be-7893-4fc4-9b26-31ce6f90f03d\") " pod="openshift-ingress/kube-auth-proxy-769f9dd78c-k2r9q" Jul 20 16:50:37.337329 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.337297 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6748e8be-7893-4fc4-9b26-31ce6f90f03d-tmp\") pod \"kube-auth-proxy-769f9dd78c-k2r9q\" (UID: \"6748e8be-7893-4fc4-9b26-31ce6f90f03d\") " pod="openshift-ingress/kube-auth-proxy-769f9dd78c-k2r9q" Jul 20 16:50:37.337598 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.337576 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6748e8be-7893-4fc4-9b26-31ce6f90f03d-tls-certs\") pod \"kube-auth-proxy-769f9dd78c-k2r9q\" (UID: \"6748e8be-7893-4fc4-9b26-31ce6f90f03d\") " pod="openshift-ingress/kube-auth-proxy-769f9dd78c-k2r9q" Jul 20 16:50:37.353139 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.353068 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbn8s\" (UniqueName: \"kubernetes.io/projected/6748e8be-7893-4fc4-9b26-31ce6f90f03d-kube-api-access-vbn8s\") pod \"kube-auth-proxy-769f9dd78c-k2r9q\" (UID: \"6748e8be-7893-4fc4-9b26-31ce6f90f03d\") " pod="openshift-ingress/kube-auth-proxy-769f9dd78c-k2r9q" Jul 20 16:50:37.416124 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.416085 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-769f9dd78c-k2r9q" Jul 20 16:50:37.651424 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.651400 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-769f9dd78c-k2r9q"] Jul 20 16:50:37.655280 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:50:37.655238 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6748e8be_7893_4fc4_9b26_31ce6f90f03d.slice/crio-889030dc6dec810d90a5cf83f1f8aa4ee75130a555131fc13562116d516680f6 WatchSource:0}: Error finding container 889030dc6dec810d90a5cf83f1f8aa4ee75130a555131fc13562116d516680f6: Status 404 returned error can't find the container with id 889030dc6dec810d90a5cf83f1f8aa4ee75130a555131fc13562116d516680f6 Jul 20 16:50:37.658275 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:37.658254 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 16:50:38.276815 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:38.276751 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" event={"ID":"0744defc-d845-428d-b757-60f6204a9d57","Type":"ContainerStarted","Data":"9a78fd06c66e02d3fe5919112e1beb8bd064486c1ccfeb0c065bca272826a22c"} Jul 20 16:50:38.277323 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:38.276851 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" Jul 20 16:50:38.278148 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:38.278114 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-769f9dd78c-k2r9q" event={"ID":"6748e8be-7893-4fc4-9b26-31ce6f90f03d","Type":"ContainerStarted","Data":"889030dc6dec810d90a5cf83f1f8aa4ee75130a555131fc13562116d516680f6"} Jul 20 16:50:38.396420 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:38.396307 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" podStartSLOduration=1.890403412 podStartE2EDuration="3.396286276s" totalImagesPullingTime="1.505882864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:50:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:50:38.395106465 +0000 UTC m=+303.181105563" watchObservedRunningTime="2026-07-20 16:50:38.396286276 +0000 UTC m=+303.182285357" Jul 20 16:50:41.295944 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:41.295908 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-769f9dd78c-k2r9q" event={"ID":"6748e8be-7893-4fc4-9b26-31ce6f90f03d","Type":"ContainerStarted","Data":"83916aa0164d24e3f73175a055739e59fa0fb073bfc60099cd263d4214364a5e"} Jul 20 16:50:41.323581 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:41.323510 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-769f9dd78c-k2r9q" podStartSLOduration=1.67956198 podStartE2EDuration="4.323486153s" totalImagesPullingTime="2.643924173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:50:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:50:41.320665005 +0000 UTC m=+306.106664085" watchObservedRunningTime="2026-07-20 16:50:41.323486153 +0000 UTC m=+306.109485232" Jul 20 16:50:49.287366 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:50:49.287335 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-86568699db-chx9m" Jul 20 16:51:53.642335 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:53.642300 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4"] Jul 20 16:51:53.645054 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:53.645031 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4" Jul 20 16:51:53.648989 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:53.648965 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 20 16:51:53.648989 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:53.648977 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 20 16:51:53.649455 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:53.649441 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-h9v4b\"" Jul 20 16:51:53.658017 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:53.657991 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4"] Jul 20 16:51:53.825001 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:53.824969 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80e2fc4c-694a-4fde-b001-c75bbbfdad92-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4\" (UID: \"80e2fc4c-694a-4fde-b001-c75bbbfdad92\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4" Jul 20 16:51:53.825001 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:53.825006 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80e2fc4c-694a-4fde-b001-c75bbbfdad92-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4\" (UID: \"80e2fc4c-694a-4fde-b001-c75bbbfdad92\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4" Jul 20 16:51:53.825235 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:53.825034 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97bwb\" (UniqueName: \"kubernetes.io/projected/80e2fc4c-694a-4fde-b001-c75bbbfdad92-kube-api-access-97bwb\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4\" (UID: \"80e2fc4c-694a-4fde-b001-c75bbbfdad92\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4" Jul 20 16:51:53.926474 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:53.926384 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80e2fc4c-694a-4fde-b001-c75bbbfdad92-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4\" (UID: \"80e2fc4c-694a-4fde-b001-c75bbbfdad92\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4" Jul 20 16:51:53.926474 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:53.926439 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80e2fc4c-694a-4fde-b001-c75bbbfdad92-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4\" (UID: \"80e2fc4c-694a-4fde-b001-c75bbbfdad92\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4" Jul 20 16:51:53.926669 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:53.926485 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97bwb\" (UniqueName: \"kubernetes.io/projected/80e2fc4c-694a-4fde-b001-c75bbbfdad92-kube-api-access-97bwb\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4\" (UID: \"80e2fc4c-694a-4fde-b001-c75bbbfdad92\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4" Jul 20 16:51:53.926899 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:53.926877 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80e2fc4c-694a-4fde-b001-c75bbbfdad92-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4\" (UID: \"80e2fc4c-694a-4fde-b001-c75bbbfdad92\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4" Jul 20 16:51:53.926954 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:53.926932 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80e2fc4c-694a-4fde-b001-c75bbbfdad92-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4\" (UID: \"80e2fc4c-694a-4fde-b001-c75bbbfdad92\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4" Jul 20 16:51:53.936186 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:53.936160 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97bwb\" (UniqueName: \"kubernetes.io/projected/80e2fc4c-694a-4fde-b001-c75bbbfdad92-kube-api-access-97bwb\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4\" (UID: \"80e2fc4c-694a-4fde-b001-c75bbbfdad92\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4" Jul 20 16:51:53.955027 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:53.955005 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4" Jul 20 16:51:54.029668 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.029631 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq"] Jul 20 16:51:54.032761 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.032741 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq" Jul 20 16:51:54.050120 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.050091 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq"] Jul 20 16:51:54.095065 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.095039 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4"] Jul 20 16:51:54.096908 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:51:54.096883 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80e2fc4c_694a_4fde_b001_c75bbbfdad92.slice/crio-91061039a7a542b2a2c9d042d231cfe66f658b92c8f1adbb57baeb94b1a68166 WatchSource:0}: Error finding container 91061039a7a542b2a2c9d042d231cfe66f658b92c8f1adbb57baeb94b1a68166: Status 404 returned error can't find the container with id 91061039a7a542b2a2c9d042d231cfe66f658b92c8f1adbb57baeb94b1a68166 Jul 20 16:51:54.229176 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.229147 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb82ff48-a85d-4311-95df-8eb56de4c627-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq\" (UID: \"cb82ff48-a85d-4311-95df-8eb56de4c627\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq" Jul 20 16:51:54.229336 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.229185 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74zld\" (UniqueName: \"kubernetes.io/projected/cb82ff48-a85d-4311-95df-8eb56de4c627-kube-api-access-74zld\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq\" (UID: \"cb82ff48-a85d-4311-95df-8eb56de4c627\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq" Jul 20 16:51:54.229336 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.229228 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb82ff48-a85d-4311-95df-8eb56de4c627-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq\" (UID: \"cb82ff48-a85d-4311-95df-8eb56de4c627\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq" Jul 20 16:51:54.330278 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.330235 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb82ff48-a85d-4311-95df-8eb56de4c627-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq\" (UID: \"cb82ff48-a85d-4311-95df-8eb56de4c627\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq" Jul 20 16:51:54.330278 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.330282 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74zld\" (UniqueName: \"kubernetes.io/projected/cb82ff48-a85d-4311-95df-8eb56de4c627-kube-api-access-74zld\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq\" (UID: \"cb82ff48-a85d-4311-95df-8eb56de4c627\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq" Jul 20 16:51:54.330526 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.330325 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb82ff48-a85d-4311-95df-8eb56de4c627-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq\" (UID: \"cb82ff48-a85d-4311-95df-8eb56de4c627\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq" Jul 20 16:51:54.330700 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.330679 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb82ff48-a85d-4311-95df-8eb56de4c627-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq\" (UID: \"cb82ff48-a85d-4311-95df-8eb56de4c627\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq" Jul 20 16:51:54.330766 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.330690 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb82ff48-a85d-4311-95df-8eb56de4c627-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq\" (UID: \"cb82ff48-a85d-4311-95df-8eb56de4c627\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq" Jul 20 16:51:54.339345 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.339319 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74zld\" (UniqueName: \"kubernetes.io/projected/cb82ff48-a85d-4311-95df-8eb56de4c627-kube-api-access-74zld\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq\" (UID: \"cb82ff48-a85d-4311-95df-8eb56de4c627\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq" Jul 20 16:51:54.345149 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.345127 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq" Jul 20 16:51:54.434843 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.434812 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2"] Jul 20 16:51:54.438133 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.438111 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2" Jul 20 16:51:54.453228 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.453194 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2"] Jul 20 16:51:54.486308 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.486284 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq"] Jul 20 16:51:54.488715 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:51:54.488688 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb82ff48_a85d_4311_95df_8eb56de4c627.slice/crio-66ddafda3f3f8c735583908d04cce2a2dce1cfb6c668191310ca99dec0dc84a1 WatchSource:0}: Error finding container 66ddafda3f3f8c735583908d04cce2a2dce1cfb6c668191310ca99dec0dc84a1: Status 404 returned error can't find the container with id 66ddafda3f3f8c735583908d04cce2a2dce1cfb6c668191310ca99dec0dc84a1 Jul 20 16:51:54.632764 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.632729 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e48ee72-a253-47ec-99ce-0da8e9bfb190-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2\" (UID: \"3e48ee72-a253-47ec-99ce-0da8e9bfb190\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2" Jul 20 16:51:54.632945 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.632814 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e48ee72-a253-47ec-99ce-0da8e9bfb190-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2\" (UID: \"3e48ee72-a253-47ec-99ce-0da8e9bfb190\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2" Jul 20 16:51:54.632945 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.632922 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4dhk\" (UniqueName: \"kubernetes.io/projected/3e48ee72-a253-47ec-99ce-0da8e9bfb190-kube-api-access-m4dhk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2\" (UID: \"3e48ee72-a253-47ec-99ce-0da8e9bfb190\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2" Jul 20 16:51:54.660483 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.660451 2566 generic.go:356] "Generic (PLEG): container finished" podID="cb82ff48-a85d-4311-95df-8eb56de4c627" containerID="5c43ef79624f2cc5dee13348176769f3e12460be97f2ea3cdacde68a464ea9be" exitCode=0 Jul 20 16:51:54.660938 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.660542 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq" event={"ID":"cb82ff48-a85d-4311-95df-8eb56de4c627","Type":"ContainerDied","Data":"5c43ef79624f2cc5dee13348176769f3e12460be97f2ea3cdacde68a464ea9be"} Jul 20 16:51:54.660938 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.660599 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq" event={"ID":"cb82ff48-a85d-4311-95df-8eb56de4c627","Type":"ContainerStarted","Data":"66ddafda3f3f8c735583908d04cce2a2dce1cfb6c668191310ca99dec0dc84a1"} Jul 20 16:51:54.662138 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.662119 2566 generic.go:356] "Generic (PLEG): container finished" podID="80e2fc4c-694a-4fde-b001-c75bbbfdad92" containerID="558b2e55f2eb2f3bf8aba9a5996de8804be0238e05bd3e922f296f924b84b0d9" exitCode=0 Jul 20 16:51:54.662232 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.662209 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4" event={"ID":"80e2fc4c-694a-4fde-b001-c75bbbfdad92","Type":"ContainerDied","Data":"558b2e55f2eb2f3bf8aba9a5996de8804be0238e05bd3e922f296f924b84b0d9"} Jul 20 16:51:54.662282 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.662251 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4" event={"ID":"80e2fc4c-694a-4fde-b001-c75bbbfdad92","Type":"ContainerStarted","Data":"91061039a7a542b2a2c9d042d231cfe66f658b92c8f1adbb57baeb94b1a68166"} Jul 20 16:51:54.734238 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.734156 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4dhk\" (UniqueName: \"kubernetes.io/projected/3e48ee72-a253-47ec-99ce-0da8e9bfb190-kube-api-access-m4dhk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2\" (UID: \"3e48ee72-a253-47ec-99ce-0da8e9bfb190\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2" Jul 20 16:51:54.734238 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.734221 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e48ee72-a253-47ec-99ce-0da8e9bfb190-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2\" (UID: \"3e48ee72-a253-47ec-99ce-0da8e9bfb190\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2" Jul 20 16:51:54.734470 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.734256 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e48ee72-a253-47ec-99ce-0da8e9bfb190-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2\" (UID: \"3e48ee72-a253-47ec-99ce-0da8e9bfb190\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2" Jul 20 16:51:54.734620 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.734596 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e48ee72-a253-47ec-99ce-0da8e9bfb190-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2\" (UID: \"3e48ee72-a253-47ec-99ce-0da8e9bfb190\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2" Jul 20 16:51:54.734677 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.734653 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e48ee72-a253-47ec-99ce-0da8e9bfb190-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2\" (UID: \"3e48ee72-a253-47ec-99ce-0da8e9bfb190\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2" Jul 20 16:51:54.747161 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.747129 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4dhk\" (UniqueName: \"kubernetes.io/projected/3e48ee72-a253-47ec-99ce-0da8e9bfb190-kube-api-access-m4dhk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2\" (UID: \"3e48ee72-a253-47ec-99ce-0da8e9bfb190\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2" Jul 20 16:51:54.747863 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.747849 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2" Jul 20 16:51:54.840516 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.840486 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq"] Jul 20 16:51:54.843527 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.843507 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq" Jul 20 16:51:54.858988 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.858956 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq"] Jul 20 16:51:54.887262 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.887236 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2"] Jul 20 16:51:54.889384 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:51:54.889346 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e48ee72_a253_47ec_99ce_0da8e9bfb190.slice/crio-8ce8b97d88eedf9b6f0097d10ed6ac325937b9b9cce8ffb7db1466938184331c WatchSource:0}: Error finding container 8ce8b97d88eedf9b6f0097d10ed6ac325937b9b9cce8ffb7db1466938184331c: Status 404 returned error can't find the container with id 8ce8b97d88eedf9b6f0097d10ed6ac325937b9b9cce8ffb7db1466938184331c Jul 20 16:51:54.937058 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.937029 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48dcbfe7-9bc6-4174-8980-befb394c0923-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq\" (UID: \"48dcbfe7-9bc6-4174-8980-befb394c0923\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq" Jul 20 16:51:54.937165 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.937092 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mf5l\" (UniqueName: \"kubernetes.io/projected/48dcbfe7-9bc6-4174-8980-befb394c0923-kube-api-access-5mf5l\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq\" (UID: \"48dcbfe7-9bc6-4174-8980-befb394c0923\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq" Jul 20 16:51:54.937213 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:54.937199 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48dcbfe7-9bc6-4174-8980-befb394c0923-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq\" (UID: \"48dcbfe7-9bc6-4174-8980-befb394c0923\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq" Jul 20 16:51:55.038405 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:55.038372 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48dcbfe7-9bc6-4174-8980-befb394c0923-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq\" (UID: \"48dcbfe7-9bc6-4174-8980-befb394c0923\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq" Jul 20 16:51:55.038571 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:55.038445 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48dcbfe7-9bc6-4174-8980-befb394c0923-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq\" (UID: \"48dcbfe7-9bc6-4174-8980-befb394c0923\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq" Jul 20 16:51:55.038571 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:55.038469 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mf5l\" (UniqueName: \"kubernetes.io/projected/48dcbfe7-9bc6-4174-8980-befb394c0923-kube-api-access-5mf5l\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq\" (UID: \"48dcbfe7-9bc6-4174-8980-befb394c0923\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq" Jul 20 16:51:55.038758 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:55.038728 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48dcbfe7-9bc6-4174-8980-befb394c0923-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq\" (UID: \"48dcbfe7-9bc6-4174-8980-befb394c0923\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq" Jul 20 16:51:55.038872 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:55.038838 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48dcbfe7-9bc6-4174-8980-befb394c0923-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq\" (UID: \"48dcbfe7-9bc6-4174-8980-befb394c0923\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq" Jul 20 16:51:55.048624 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:55.048598 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mf5l\" (UniqueName: \"kubernetes.io/projected/48dcbfe7-9bc6-4174-8980-befb394c0923-kube-api-access-5mf5l\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq\" (UID: \"48dcbfe7-9bc6-4174-8980-befb394c0923\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq" Jul 20 16:51:55.154652 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:55.154621 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq" Jul 20 16:51:55.291439 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:55.291416 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq"] Jul 20 16:51:55.293265 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:51:55.293239 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48dcbfe7_9bc6_4174_8980_befb394c0923.slice/crio-cb2d80639c909c63f78584621e44686d3e50acc0bfbc74e3e8789af596b82707 WatchSource:0}: Error finding container cb2d80639c909c63f78584621e44686d3e50acc0bfbc74e3e8789af596b82707: Status 404 returned error can't find the container with id cb2d80639c909c63f78584621e44686d3e50acc0bfbc74e3e8789af596b82707 Jul 20 16:51:55.669127 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:55.669089 2566 generic.go:356] "Generic (PLEG): container finished" podID="3e48ee72-a253-47ec-99ce-0da8e9bfb190" containerID="8f96491d76d0c71eacb03c5a05d9343c85d00d4819852d52c42bd211cc588b4b" exitCode=0 Jul 20 16:51:55.669584 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:55.669146 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2" event={"ID":"3e48ee72-a253-47ec-99ce-0da8e9bfb190","Type":"ContainerDied","Data":"8f96491d76d0c71eacb03c5a05d9343c85d00d4819852d52c42bd211cc588b4b"} Jul 20 16:51:55.669584 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:55.669210 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2" event={"ID":"3e48ee72-a253-47ec-99ce-0da8e9bfb190","Type":"ContainerStarted","Data":"8ce8b97d88eedf9b6f0097d10ed6ac325937b9b9cce8ffb7db1466938184331c"} Jul 20 16:51:55.670803 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:55.670747 2566 generic.go:356] "Generic (PLEG): container finished" podID="48dcbfe7-9bc6-4174-8980-befb394c0923" containerID="96e94478ce90941ff872e45c9a84cb380e1f70e8282f9e7e0788a59f0d587d6f" exitCode=0 Jul 20 16:51:55.670924 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:55.670850 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq" event={"ID":"48dcbfe7-9bc6-4174-8980-befb394c0923","Type":"ContainerDied","Data":"96e94478ce90941ff872e45c9a84cb380e1f70e8282f9e7e0788a59f0d587d6f"} Jul 20 16:51:55.670924 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:55.670901 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq" event={"ID":"48dcbfe7-9bc6-4174-8980-befb394c0923","Type":"ContainerStarted","Data":"cb2d80639c909c63f78584621e44686d3e50acc0bfbc74e3e8789af596b82707"} Jul 20 16:51:55.673000 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:55.672980 2566 generic.go:356] "Generic (PLEG): container finished" podID="cb82ff48-a85d-4311-95df-8eb56de4c627" containerID="cca5fc4fd26c309f71751e5e24615c7233d52114e28bd97d894efce0d73a4a69" exitCode=0 Jul 20 16:51:55.673067 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:55.673010 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq" event={"ID":"cb82ff48-a85d-4311-95df-8eb56de4c627","Type":"ContainerDied","Data":"cca5fc4fd26c309f71751e5e24615c7233d52114e28bd97d894efce0d73a4a69"} Jul 20 16:51:56.681222 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:56.681174 2566 generic.go:356] "Generic (PLEG): container finished" podID="cb82ff48-a85d-4311-95df-8eb56de4c627" containerID="2f17f3559284802f059a792697ac63379bb23390e0bf5821ba8b7aee10bb9ede" exitCode=0 Jul 20 16:51:56.681528 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:56.681261 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq" event={"ID":"cb82ff48-a85d-4311-95df-8eb56de4c627","Type":"ContainerDied","Data":"2f17f3559284802f059a792697ac63379bb23390e0bf5821ba8b7aee10bb9ede"} Jul 20 16:51:57.692141 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:57.692100 2566 generic.go:356] "Generic (PLEG): container finished" podID="3e48ee72-a253-47ec-99ce-0da8e9bfb190" containerID="b03c4b91b6196c68dc41ab8e5236dcc2af041895a9f905dab86b616c3999ca70" exitCode=0 Jul 20 16:51:57.692531 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:57.692184 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2" event={"ID":"3e48ee72-a253-47ec-99ce-0da8e9bfb190","Type":"ContainerDied","Data":"b03c4b91b6196c68dc41ab8e5236dcc2af041895a9f905dab86b616c3999ca70"} Jul 20 16:51:57.694047 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:57.694013 2566 generic.go:356] "Generic (PLEG): container finished" podID="80e2fc4c-694a-4fde-b001-c75bbbfdad92" containerID="7541669c0bb991c075727459304446a496e00c0842b8ae1baea9f9a2b1ef9189" exitCode=0 Jul 20 16:51:57.694151 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:57.694101 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4" event={"ID":"80e2fc4c-694a-4fde-b001-c75bbbfdad92","Type":"ContainerDied","Data":"7541669c0bb991c075727459304446a496e00c0842b8ae1baea9f9a2b1ef9189"} Jul 20 16:51:57.845650 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:57.845618 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq" Jul 20 16:51:57.859488 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:57.859460 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74zld\" (UniqueName: \"kubernetes.io/projected/cb82ff48-a85d-4311-95df-8eb56de4c627-kube-api-access-74zld\") pod \"cb82ff48-a85d-4311-95df-8eb56de4c627\" (UID: \"cb82ff48-a85d-4311-95df-8eb56de4c627\") " Jul 20 16:51:57.859628 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:57.859613 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb82ff48-a85d-4311-95df-8eb56de4c627-util\") pod \"cb82ff48-a85d-4311-95df-8eb56de4c627\" (UID: \"cb82ff48-a85d-4311-95df-8eb56de4c627\") " Jul 20 16:51:57.859689 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:57.859679 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb82ff48-a85d-4311-95df-8eb56de4c627-bundle\") pod \"cb82ff48-a85d-4311-95df-8eb56de4c627\" (UID: \"cb82ff48-a85d-4311-95df-8eb56de4c627\") " Jul 20 16:51:57.860166 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:57.860137 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb82ff48-a85d-4311-95df-8eb56de4c627-bundle" (OuterVolumeSpecName: "bundle") pod "cb82ff48-a85d-4311-95df-8eb56de4c627" (UID: "cb82ff48-a85d-4311-95df-8eb56de4c627"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:51:57.862439 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:57.862403 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb82ff48-a85d-4311-95df-8eb56de4c627-kube-api-access-74zld" (OuterVolumeSpecName: "kube-api-access-74zld") pod "cb82ff48-a85d-4311-95df-8eb56de4c627" (UID: "cb82ff48-a85d-4311-95df-8eb56de4c627"). InnerVolumeSpecName "kube-api-access-74zld". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:51:57.866222 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:57.866193 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb82ff48-a85d-4311-95df-8eb56de4c627-util" (OuterVolumeSpecName: "util") pod "cb82ff48-a85d-4311-95df-8eb56de4c627" (UID: "cb82ff48-a85d-4311-95df-8eb56de4c627"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:51:57.961206 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:57.961136 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb82ff48-a85d-4311-95df-8eb56de4c627-bundle\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:51:57.961206 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:57.961163 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74zld\" (UniqueName: \"kubernetes.io/projected/cb82ff48-a85d-4311-95df-8eb56de4c627-kube-api-access-74zld\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:51:57.961206 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:57.961172 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb82ff48-a85d-4311-95df-8eb56de4c627-util\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:51:58.700573 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:58.700541 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq" event={"ID":"cb82ff48-a85d-4311-95df-8eb56de4c627","Type":"ContainerDied","Data":"66ddafda3f3f8c735583908d04cce2a2dce1cfb6c668191310ca99dec0dc84a1"} Jul 20 16:51:58.701011 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:58.700592 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66ddafda3f3f8c735583908d04cce2a2dce1cfb6c668191310ca99dec0dc84a1" Jul 20 16:51:58.701011 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:58.700557 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq" Jul 20 16:51:58.702879 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:58.702855 2566 generic.go:356] "Generic (PLEG): container finished" podID="3e48ee72-a253-47ec-99ce-0da8e9bfb190" containerID="d0a6aa1af997f68c9ea31d3e1eda16e3ea46c81c54c3c667bc1c38ba8669bd6a" exitCode=0 Jul 20 16:51:58.702977 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:58.702948 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2" event={"ID":"3e48ee72-a253-47ec-99ce-0da8e9bfb190","Type":"ContainerDied","Data":"d0a6aa1af997f68c9ea31d3e1eda16e3ea46c81c54c3c667bc1c38ba8669bd6a"} Jul 20 16:51:58.705051 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:58.705029 2566 generic.go:356] "Generic (PLEG): container finished" podID="80e2fc4c-694a-4fde-b001-c75bbbfdad92" containerID="ad177d952300fef8f50002fca7e45ce6bcd819d34eebb48058951772c1aa63ec" exitCode=0 Jul 20 16:51:58.705141 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:58.705120 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4" event={"ID":"80e2fc4c-694a-4fde-b001-c75bbbfdad92","Type":"ContainerDied","Data":"ad177d952300fef8f50002fca7e45ce6bcd819d34eebb48058951772c1aa63ec"} Jul 20 16:51:59.712551 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:59.712522 2566 generic.go:356] "Generic (PLEG): container finished" podID="48dcbfe7-9bc6-4174-8980-befb394c0923" containerID="97da6c9a7328334d54d8f140d46637063c69b96b6d6a7f63ebcac71d2e119dd6" exitCode=0 Jul 20 16:51:59.712942 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:59.712615 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq" event={"ID":"48dcbfe7-9bc6-4174-8980-befb394c0923","Type":"ContainerDied","Data":"97da6c9a7328334d54d8f140d46637063c69b96b6d6a7f63ebcac71d2e119dd6"} Jul 20 16:51:59.847990 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:59.847967 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4" Jul 20 16:51:59.877119 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:59.877091 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80e2fc4c-694a-4fde-b001-c75bbbfdad92-bundle\") pod \"80e2fc4c-694a-4fde-b001-c75bbbfdad92\" (UID: \"80e2fc4c-694a-4fde-b001-c75bbbfdad92\") " Jul 20 16:51:59.877250 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:59.877164 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97bwb\" (UniqueName: \"kubernetes.io/projected/80e2fc4c-694a-4fde-b001-c75bbbfdad92-kube-api-access-97bwb\") pod \"80e2fc4c-694a-4fde-b001-c75bbbfdad92\" (UID: \"80e2fc4c-694a-4fde-b001-c75bbbfdad92\") " Jul 20 16:51:59.877322 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:59.877296 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80e2fc4c-694a-4fde-b001-c75bbbfdad92-util\") pod \"80e2fc4c-694a-4fde-b001-c75bbbfdad92\" (UID: \"80e2fc4c-694a-4fde-b001-c75bbbfdad92\") " Jul 20 16:51:59.878304 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:59.877939 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80e2fc4c-694a-4fde-b001-c75bbbfdad92-bundle" (OuterVolumeSpecName: "bundle") pod "80e2fc4c-694a-4fde-b001-c75bbbfdad92" (UID: "80e2fc4c-694a-4fde-b001-c75bbbfdad92"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:51:59.880057 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:59.879983 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80e2fc4c-694a-4fde-b001-c75bbbfdad92-kube-api-access-97bwb" (OuterVolumeSpecName: "kube-api-access-97bwb") pod "80e2fc4c-694a-4fde-b001-c75bbbfdad92" (UID: "80e2fc4c-694a-4fde-b001-c75bbbfdad92"). InnerVolumeSpecName "kube-api-access-97bwb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:51:59.882485 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:59.882458 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80e2fc4c-694a-4fde-b001-c75bbbfdad92-util" (OuterVolumeSpecName: "util") pod "80e2fc4c-694a-4fde-b001-c75bbbfdad92" (UID: "80e2fc4c-694a-4fde-b001-c75bbbfdad92"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:51:59.882601 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:59.882498 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2" Jul 20 16:51:59.978796 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:59.978691 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e48ee72-a253-47ec-99ce-0da8e9bfb190-bundle\") pod \"3e48ee72-a253-47ec-99ce-0da8e9bfb190\" (UID: \"3e48ee72-a253-47ec-99ce-0da8e9bfb190\") " Jul 20 16:51:59.978936 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:59.978865 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e48ee72-a253-47ec-99ce-0da8e9bfb190-util\") pod \"3e48ee72-a253-47ec-99ce-0da8e9bfb190\" (UID: \"3e48ee72-a253-47ec-99ce-0da8e9bfb190\") " Jul 20 16:51:59.978936 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:59.978915 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4dhk\" (UniqueName: \"kubernetes.io/projected/3e48ee72-a253-47ec-99ce-0da8e9bfb190-kube-api-access-m4dhk\") pod \"3e48ee72-a253-47ec-99ce-0da8e9bfb190\" (UID: \"3e48ee72-a253-47ec-99ce-0da8e9bfb190\") " Jul 20 16:51:59.979177 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:59.979162 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97bwb\" (UniqueName: \"kubernetes.io/projected/80e2fc4c-694a-4fde-b001-c75bbbfdad92-kube-api-access-97bwb\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:51:59.979177 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:59.979165 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e48ee72-a253-47ec-99ce-0da8e9bfb190-bundle" (OuterVolumeSpecName: "bundle") pod "3e48ee72-a253-47ec-99ce-0da8e9bfb190" (UID: "3e48ee72-a253-47ec-99ce-0da8e9bfb190"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:51:59.979253 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:59.979180 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80e2fc4c-694a-4fde-b001-c75bbbfdad92-util\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:51:59.979253 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:59.979194 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80e2fc4c-694a-4fde-b001-c75bbbfdad92-bundle\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:51:59.981103 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:51:59.981081 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e48ee72-a253-47ec-99ce-0da8e9bfb190-kube-api-access-m4dhk" (OuterVolumeSpecName: "kube-api-access-m4dhk") pod "3e48ee72-a253-47ec-99ce-0da8e9bfb190" (UID: "3e48ee72-a253-47ec-99ce-0da8e9bfb190"). InnerVolumeSpecName "kube-api-access-m4dhk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:52:00.035949 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:00.035912 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e48ee72-a253-47ec-99ce-0da8e9bfb190-util" (OuterVolumeSpecName: "util") pod "3e48ee72-a253-47ec-99ce-0da8e9bfb190" (UID: "3e48ee72-a253-47ec-99ce-0da8e9bfb190"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:52:00.080561 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:00.080530 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e48ee72-a253-47ec-99ce-0da8e9bfb190-bundle\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:52:00.080561 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:00.080556 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e48ee72-a253-47ec-99ce-0da8e9bfb190-util\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:52:00.080561 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:00.080565 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4dhk\" (UniqueName: \"kubernetes.io/projected/3e48ee72-a253-47ec-99ce-0da8e9bfb190-kube-api-access-m4dhk\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:52:00.725514 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:00.725486 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2" Jul 20 16:52:00.725936 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:00.725512 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2" event={"ID":"3e48ee72-a253-47ec-99ce-0da8e9bfb190","Type":"ContainerDied","Data":"8ce8b97d88eedf9b6f0097d10ed6ac325937b9b9cce8ffb7db1466938184331c"} Jul 20 16:52:00.725936 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:00.725590 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ce8b97d88eedf9b6f0097d10ed6ac325937b9b9cce8ffb7db1466938184331c" Jul 20 16:52:00.727506 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:00.727482 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4" event={"ID":"80e2fc4c-694a-4fde-b001-c75bbbfdad92","Type":"ContainerDied","Data":"91061039a7a542b2a2c9d042d231cfe66f658b92c8f1adbb57baeb94b1a68166"} Jul 20 16:52:00.727638 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:00.727521 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91061039a7a542b2a2c9d042d231cfe66f658b92c8f1adbb57baeb94b1a68166" Jul 20 16:52:00.727638 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:00.727485 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4" Jul 20 16:52:00.729859 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:00.729835 2566 generic.go:356] "Generic (PLEG): container finished" podID="48dcbfe7-9bc6-4174-8980-befb394c0923" containerID="63c33792d837cb5f461b1b598efc92733e342b6259c094c8be1ed1a28ccb1bba" exitCode=0 Jul 20 16:52:00.729985 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:00.729923 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq" event={"ID":"48dcbfe7-9bc6-4174-8980-befb394c0923","Type":"ContainerDied","Data":"63c33792d837cb5f461b1b598efc92733e342b6259c094c8be1ed1a28ccb1bba"} Jul 20 16:52:01.862575 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:01.862553 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq" Jul 20 16:52:01.894433 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:01.894407 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48dcbfe7-9bc6-4174-8980-befb394c0923-util\") pod \"48dcbfe7-9bc6-4174-8980-befb394c0923\" (UID: \"48dcbfe7-9bc6-4174-8980-befb394c0923\") " Jul 20 16:52:01.894584 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:01.894445 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48dcbfe7-9bc6-4174-8980-befb394c0923-bundle\") pod \"48dcbfe7-9bc6-4174-8980-befb394c0923\" (UID: \"48dcbfe7-9bc6-4174-8980-befb394c0923\") " Jul 20 16:52:01.894584 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:01.894470 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mf5l\" (UniqueName: \"kubernetes.io/projected/48dcbfe7-9bc6-4174-8980-befb394c0923-kube-api-access-5mf5l\") pod \"48dcbfe7-9bc6-4174-8980-befb394c0923\" (UID: \"48dcbfe7-9bc6-4174-8980-befb394c0923\") " Jul 20 16:52:01.894968 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:01.894944 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48dcbfe7-9bc6-4174-8980-befb394c0923-bundle" (OuterVolumeSpecName: "bundle") pod "48dcbfe7-9bc6-4174-8980-befb394c0923" (UID: "48dcbfe7-9bc6-4174-8980-befb394c0923"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:52:01.896959 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:01.896932 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48dcbfe7-9bc6-4174-8980-befb394c0923-kube-api-access-5mf5l" (OuterVolumeSpecName: "kube-api-access-5mf5l") pod "48dcbfe7-9bc6-4174-8980-befb394c0923" (UID: "48dcbfe7-9bc6-4174-8980-befb394c0923"). InnerVolumeSpecName "kube-api-access-5mf5l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:52:01.898566 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:01.898544 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48dcbfe7-9bc6-4174-8980-befb394c0923-util" (OuterVolumeSpecName: "util") pod "48dcbfe7-9bc6-4174-8980-befb394c0923" (UID: "48dcbfe7-9bc6-4174-8980-befb394c0923"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:52:01.995930 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:01.995861 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48dcbfe7-9bc6-4174-8980-befb394c0923-util\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:52:01.995930 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:01.995885 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48dcbfe7-9bc6-4174-8980-befb394c0923-bundle\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:52:01.995930 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:01.995894 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mf5l\" (UniqueName: \"kubernetes.io/projected/48dcbfe7-9bc6-4174-8980-befb394c0923-kube-api-access-5mf5l\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:52:02.742754 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:02.742717 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq" Jul 20 16:52:02.742955 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:02.742722 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq" event={"ID":"48dcbfe7-9bc6-4174-8980-befb394c0923","Type":"ContainerDied","Data":"cb2d80639c909c63f78584621e44686d3e50acc0bfbc74e3e8789af596b82707"} Jul 20 16:52:02.742955 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:02.742832 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb2d80639c909c63f78584621e44686d3e50acc0bfbc74e3e8789af596b82707" Jul 20 16:52:08.326330 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326291 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c8b5fd8c9-fsrqc"] Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326634 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb82ff48-a85d-4311-95df-8eb56de4c627" containerName="extract" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326645 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb82ff48-a85d-4311-95df-8eb56de4c627" containerName="extract" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326657 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80e2fc4c-694a-4fde-b001-c75bbbfdad92" containerName="util" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326662 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="80e2fc4c-694a-4fde-b001-c75bbbfdad92" containerName="util" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326667 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e48ee72-a253-47ec-99ce-0da8e9bfb190" containerName="extract" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326672 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e48ee72-a253-47ec-99ce-0da8e9bfb190" containerName="extract" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326682 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80e2fc4c-694a-4fde-b001-c75bbbfdad92" containerName="extract" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326687 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="80e2fc4c-694a-4fde-b001-c75bbbfdad92" containerName="extract" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326694 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e48ee72-a253-47ec-99ce-0da8e9bfb190" containerName="pull" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326699 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e48ee72-a253-47ec-99ce-0da8e9bfb190" containerName="pull" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326709 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80e2fc4c-694a-4fde-b001-c75bbbfdad92" containerName="pull" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326714 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="80e2fc4c-694a-4fde-b001-c75bbbfdad92" containerName="pull" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326722 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48dcbfe7-9bc6-4174-8980-befb394c0923" containerName="extract" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326727 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="48dcbfe7-9bc6-4174-8980-befb394c0923" containerName="extract" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326736 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb82ff48-a85d-4311-95df-8eb56de4c627" containerName="util" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326740 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb82ff48-a85d-4311-95df-8eb56de4c627" containerName="util" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326747 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48dcbfe7-9bc6-4174-8980-befb394c0923" containerName="util" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326752 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="48dcbfe7-9bc6-4174-8980-befb394c0923" containerName="util" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326757 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb82ff48-a85d-4311-95df-8eb56de4c627" containerName="pull" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326762 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb82ff48-a85d-4311-95df-8eb56de4c627" containerName="pull" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326785 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e48ee72-a253-47ec-99ce-0da8e9bfb190" containerName="util" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326792 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e48ee72-a253-47ec-99ce-0da8e9bfb190" containerName="util" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326801 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48dcbfe7-9bc6-4174-8980-befb394c0923" containerName="pull" Jul 20 16:52:08.326799 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326809 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="48dcbfe7-9bc6-4174-8980-befb394c0923" containerName="pull" Jul 20 16:52:08.327481 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326885 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="80e2fc4c-694a-4fde-b001-c75bbbfdad92" containerName="extract" Jul 20 16:52:08.327481 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326898 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="48dcbfe7-9bc6-4174-8980-befb394c0923" containerName="extract" Jul 20 16:52:08.327481 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326904 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb82ff48-a85d-4311-95df-8eb56de4c627" containerName="extract" Jul 20 16:52:08.327481 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.326910 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e48ee72-a253-47ec-99ce-0da8e9bfb190" containerName="extract" Jul 20 16:52:08.330012 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.329996 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.350511 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.350483 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c8b5fd8c9-fsrqc"] Jul 20 16:52:08.356518 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.356494 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f267b375-354c-444b-8484-8465c41f89d0-trusted-ca-bundle\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.356658 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.356536 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f267b375-354c-444b-8484-8465c41f89d0-console-serving-cert\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.356658 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.356563 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbnqw\" (UniqueName: \"kubernetes.io/projected/f267b375-354c-444b-8484-8465c41f89d0-kube-api-access-fbnqw\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.356658 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.356626 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f267b375-354c-444b-8484-8465c41f89d0-console-config\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.356814 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.356756 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f267b375-354c-444b-8484-8465c41f89d0-oauth-serving-cert\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.356858 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.356845 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f267b375-354c-444b-8484-8465c41f89d0-service-ca\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.356906 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.356881 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f267b375-354c-444b-8484-8465c41f89d0-console-oauth-config\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.457585 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.457552 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f267b375-354c-444b-8484-8465c41f89d0-trusted-ca-bundle\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.457826 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.457591 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f267b375-354c-444b-8484-8465c41f89d0-console-serving-cert\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.457826 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.457625 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbnqw\" (UniqueName: \"kubernetes.io/projected/f267b375-354c-444b-8484-8465c41f89d0-kube-api-access-fbnqw\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.457826 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.457655 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f267b375-354c-444b-8484-8465c41f89d0-console-config\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.457826 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.457691 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f267b375-354c-444b-8484-8465c41f89d0-oauth-serving-cert\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.457826 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.457710 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f267b375-354c-444b-8484-8465c41f89d0-service-ca\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.457826 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.457730 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f267b375-354c-444b-8484-8465c41f89d0-console-oauth-config\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.458548 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.458522 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f267b375-354c-444b-8484-8465c41f89d0-oauth-serving-cert\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.458641 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.458566 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f267b375-354c-444b-8484-8465c41f89d0-console-config\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.458845 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.458820 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f267b375-354c-444b-8484-8465c41f89d0-service-ca\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.458944 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.458922 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f267b375-354c-444b-8484-8465c41f89d0-trusted-ca-bundle\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.460359 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.460333 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f267b375-354c-444b-8484-8465c41f89d0-console-oauth-config\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.460439 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.460417 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f267b375-354c-444b-8484-8465c41f89d0-console-serving-cert\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.467548 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.467524 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbnqw\" (UniqueName: \"kubernetes.io/projected/f267b375-354c-444b-8484-8465c41f89d0-kube-api-access-fbnqw\") pod \"console-5c8b5fd8c9-fsrqc\" (UID: \"f267b375-354c-444b-8484-8465c41f89d0\") " pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.639738 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.639702 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:08.779441 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:08.779421 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c8b5fd8c9-fsrqc"] Jul 20 16:52:08.781561 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:52:08.781526 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf267b375_354c_444b_8484_8465c41f89d0.slice/crio-e8a359868a8b4f132dd7710add4eb6900d6ba9efb67cebe159e8ec62e55b25a5 WatchSource:0}: Error finding container e8a359868a8b4f132dd7710add4eb6900d6ba9efb67cebe159e8ec62e55b25a5: Status 404 returned error can't find the container with id e8a359868a8b4f132dd7710add4eb6900d6ba9efb67cebe159e8ec62e55b25a5 Jul 20 16:52:09.782300 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:09.782267 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c8b5fd8c9-fsrqc" event={"ID":"f267b375-354c-444b-8484-8465c41f89d0","Type":"ContainerStarted","Data":"d99d8e82197bf05d770d954f7a2ddbf236283fcb22d3893dd9d3420704746104"} Jul 20 16:52:09.782802 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:09.782322 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c8b5fd8c9-fsrqc" event={"ID":"f267b375-354c-444b-8484-8465c41f89d0","Type":"ContainerStarted","Data":"e8a359868a8b4f132dd7710add4eb6900d6ba9efb67cebe159e8ec62e55b25a5"} Jul 20 16:52:09.829719 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:09.829655 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c8b5fd8c9-fsrqc" podStartSLOduration=1.829620415 podStartE2EDuration="1.829620415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:52:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:52:09.828041053 +0000 UTC m=+394.614040132" watchObservedRunningTime="2026-07-20 16:52:09.829620415 +0000 UTC m=+394.615619498" Jul 20 16:52:18.640750 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:18.640709 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:18.641224 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:18.640836 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:18.645976 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:18.645952 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:18.834582 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:18.834551 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c8b5fd8c9-fsrqc" Jul 20 16:52:18.903524 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:18.903443 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c8b758c77-gmbmn"] Jul 20 16:52:43.925428 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:43.925390 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c8b758c77-gmbmn" podUID="d872c757-66fb-4b42-a5eb-a1660f7834f6" containerName="console" containerID="cri-o://40ccb96266de98be63740646bb6a655b511c82cb9b12dc2e01dae447ee67cbfa" gracePeriod=15 Jul 20 16:52:44.174047 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.174023 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:52:44.193203 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.193130 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-trusted-ca-bundle\") pod \"d872c757-66fb-4b42-a5eb-a1660f7834f6\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " Jul 20 16:52:44.193203 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.193174 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-oauth-serving-cert\") pod \"d872c757-66fb-4b42-a5eb-a1660f7834f6\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " Jul 20 16:52:44.193203 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.193201 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-service-ca\") pod \"d872c757-66fb-4b42-a5eb-a1660f7834f6\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " Jul 20 16:52:44.193418 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.193312 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d872c757-66fb-4b42-a5eb-a1660f7834f6-console-serving-cert\") pod \"d872c757-66fb-4b42-a5eb-a1660f7834f6\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " Jul 20 16:52:44.193418 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.193348 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d872c757-66fb-4b42-a5eb-a1660f7834f6-console-oauth-config\") pod \"d872c757-66fb-4b42-a5eb-a1660f7834f6\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " Jul 20 16:52:44.193418 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.193402 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmfh4\" (UniqueName: \"kubernetes.io/projected/d872c757-66fb-4b42-a5eb-a1660f7834f6-kube-api-access-lmfh4\") pod \"d872c757-66fb-4b42-a5eb-a1660f7834f6\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " Jul 20 16:52:44.193564 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.193432 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-console-config\") pod \"d872c757-66fb-4b42-a5eb-a1660f7834f6\" (UID: \"d872c757-66fb-4b42-a5eb-a1660f7834f6\") " Jul 20 16:52:44.193564 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.193531 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d872c757-66fb-4b42-a5eb-a1660f7834f6" (UID: "d872c757-66fb-4b42-a5eb-a1660f7834f6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:52:44.193664 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.193561 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-service-ca" (OuterVolumeSpecName: "service-ca") pod "d872c757-66fb-4b42-a5eb-a1660f7834f6" (UID: "d872c757-66fb-4b42-a5eb-a1660f7834f6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:52:44.193664 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.193570 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d872c757-66fb-4b42-a5eb-a1660f7834f6" (UID: "d872c757-66fb-4b42-a5eb-a1660f7834f6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:52:44.193896 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.193879 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-trusted-ca-bundle\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:52:44.193949 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.193902 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-oauth-serving-cert\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:52:44.193949 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.193916 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-service-ca\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:52:44.193949 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.193936 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-console-config" (OuterVolumeSpecName: "console-config") pod "d872c757-66fb-4b42-a5eb-a1660f7834f6" (UID: "d872c757-66fb-4b42-a5eb-a1660f7834f6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:52:44.195575 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.195547 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d872c757-66fb-4b42-a5eb-a1660f7834f6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d872c757-66fb-4b42-a5eb-a1660f7834f6" (UID: "d872c757-66fb-4b42-a5eb-a1660f7834f6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:52:44.195676 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.195646 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d872c757-66fb-4b42-a5eb-a1660f7834f6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d872c757-66fb-4b42-a5eb-a1660f7834f6" (UID: "d872c757-66fb-4b42-a5eb-a1660f7834f6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:52:44.195735 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.195714 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d872c757-66fb-4b42-a5eb-a1660f7834f6-kube-api-access-lmfh4" (OuterVolumeSpecName: "kube-api-access-lmfh4") pod "d872c757-66fb-4b42-a5eb-a1660f7834f6" (UID: "d872c757-66fb-4b42-a5eb-a1660f7834f6"). InnerVolumeSpecName "kube-api-access-lmfh4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:52:44.294761 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.294722 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmfh4\" (UniqueName: \"kubernetes.io/projected/d872c757-66fb-4b42-a5eb-a1660f7834f6-kube-api-access-lmfh4\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:52:44.294761 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.294754 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d872c757-66fb-4b42-a5eb-a1660f7834f6-console-config\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:52:44.294761 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.294766 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d872c757-66fb-4b42-a5eb-a1660f7834f6-console-serving-cert\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:52:44.294761 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.294795 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d872c757-66fb-4b42-a5eb-a1660f7834f6-console-oauth-config\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:52:44.972165 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.972135 2566 generic.go:356] "Generic (PLEG): container finished" podID="d872c757-66fb-4b42-a5eb-a1660f7834f6" containerID="40ccb96266de98be63740646bb6a655b511c82cb9b12dc2e01dae447ee67cbfa" exitCode=0 Jul 20 16:52:44.972540 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.972181 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c8b758c77-gmbmn" Jul 20 16:52:44.972540 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.972221 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c8b758c77-gmbmn" event={"ID":"d872c757-66fb-4b42-a5eb-a1660f7834f6","Type":"ContainerDied","Data":"40ccb96266de98be63740646bb6a655b511c82cb9b12dc2e01dae447ee67cbfa"} Jul 20 16:52:44.972540 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.972272 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c8b758c77-gmbmn" event={"ID":"d872c757-66fb-4b42-a5eb-a1660f7834f6","Type":"ContainerDied","Data":"0443fe43746acba0def277196bf6a3dfd3bc7606b06f216e4e2d467aaad5e07a"} Jul 20 16:52:44.972540 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.972294 2566 scope.go:117] "RemoveContainer" containerID="40ccb96266de98be63740646bb6a655b511c82cb9b12dc2e01dae447ee67cbfa" Jul 20 16:52:44.981271 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.981251 2566 scope.go:117] "RemoveContainer" containerID="40ccb96266de98be63740646bb6a655b511c82cb9b12dc2e01dae447ee67cbfa" Jul 20 16:52:44.981522 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:52:44.981505 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40ccb96266de98be63740646bb6a655b511c82cb9b12dc2e01dae447ee67cbfa\": container with ID starting with 40ccb96266de98be63740646bb6a655b511c82cb9b12dc2e01dae447ee67cbfa not found: ID does not exist" containerID="40ccb96266de98be63740646bb6a655b511c82cb9b12dc2e01dae447ee67cbfa" Jul 20 16:52:44.981571 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.981530 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40ccb96266de98be63740646bb6a655b511c82cb9b12dc2e01dae447ee67cbfa"} err="failed to get container status \"40ccb96266de98be63740646bb6a655b511c82cb9b12dc2e01dae447ee67cbfa\": rpc error: code = NotFound desc = could not find container \"40ccb96266de98be63740646bb6a655b511c82cb9b12dc2e01dae447ee67cbfa\": container with ID starting with 40ccb96266de98be63740646bb6a655b511c82cb9b12dc2e01dae447ee67cbfa not found: ID does not exist" Jul 20 16:52:44.995327 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.995299 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c8b758c77-gmbmn"] Jul 20 16:52:44.997242 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:44.997223 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c8b758c77-gmbmn"] Jul 20 16:52:45.814831 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:52:45.814798 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d872c757-66fb-4b42-a5eb-a1660f7834f6" path="/var/lib/kubelet/pods/d872c757-66fb-4b42-a5eb-a1660f7834f6/volumes" Jul 20 16:53:17.966064 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:17.965983 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rwdxc"] Jul 20 16:53:17.966500 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:17.966389 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d872c757-66fb-4b42-a5eb-a1660f7834f6" containerName="console" Jul 20 16:53:17.966500 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:17.966403 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d872c757-66fb-4b42-a5eb-a1660f7834f6" containerName="console" Jul 20 16:53:17.966500 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:17.966476 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d872c757-66fb-4b42-a5eb-a1660f7834f6" containerName="console" Jul 20 16:53:17.969278 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:17.969260 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rwdxc" Jul 20 16:53:17.971600 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:17.971575 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-h9v4b\"" Jul 20 16:53:17.971600 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:17.971590 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 20 16:53:17.971888 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:17.971872 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 20 16:53:17.972127 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:17.972114 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 20 16:53:17.986854 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:17.986825 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rwdxc"] Jul 20 16:53:18.072418 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:18.072384 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rwdxc"] Jul 20 16:53:18.089174 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:18.089145 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzl47\" (UniqueName: \"kubernetes.io/projected/6e57b695-2dc8-436a-b18f-6236035f7214-kube-api-access-nzl47\") pod \"limitador-limitador-64c94fd64f-rwdxc\" (UID: \"6e57b695-2dc8-436a-b18f-6236035f7214\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rwdxc" Jul 20 16:53:18.089325 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:18.089205 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6e57b695-2dc8-436a-b18f-6236035f7214-config-file\") pod \"limitador-limitador-64c94fd64f-rwdxc\" (UID: \"6e57b695-2dc8-436a-b18f-6236035f7214\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rwdxc" Jul 20 16:53:18.189831 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:18.189792 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6e57b695-2dc8-436a-b18f-6236035f7214-config-file\") pod \"limitador-limitador-64c94fd64f-rwdxc\" (UID: \"6e57b695-2dc8-436a-b18f-6236035f7214\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rwdxc" Jul 20 16:53:18.189997 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:18.189862 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzl47\" (UniqueName: \"kubernetes.io/projected/6e57b695-2dc8-436a-b18f-6236035f7214-kube-api-access-nzl47\") pod \"limitador-limitador-64c94fd64f-rwdxc\" (UID: \"6e57b695-2dc8-436a-b18f-6236035f7214\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rwdxc" Jul 20 16:53:18.190398 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:18.190377 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6e57b695-2dc8-436a-b18f-6236035f7214-config-file\") pod \"limitador-limitador-64c94fd64f-rwdxc\" (UID: \"6e57b695-2dc8-436a-b18f-6236035f7214\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rwdxc" Jul 20 16:53:18.204293 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:18.204263 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzl47\" (UniqueName: \"kubernetes.io/projected/6e57b695-2dc8-436a-b18f-6236035f7214-kube-api-access-nzl47\") pod \"limitador-limitador-64c94fd64f-rwdxc\" (UID: \"6e57b695-2dc8-436a-b18f-6236035f7214\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rwdxc" Jul 20 16:53:18.279415 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:18.279320 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rwdxc" Jul 20 16:53:18.423627 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:53:18.423602 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e57b695_2dc8_436a_b18f_6236035f7214.slice/crio-89cbe8ab5608219dac1bf60b7645435c73ae390ee67df1f20584431a346cc95f WatchSource:0}: Error finding container 89cbe8ab5608219dac1bf60b7645435c73ae390ee67df1f20584431a346cc95f: Status 404 returned error can't find the container with id 89cbe8ab5608219dac1bf60b7645435c73ae390ee67df1f20584431a346cc95f Jul 20 16:53:18.427608 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:18.427582 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rwdxc"] Jul 20 16:53:19.102791 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:19.102732 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-k9tmt"] Jul 20 16:53:19.106734 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:19.106709 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-k9tmt" Jul 20 16:53:19.110729 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:19.110692 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gcwqs\"" Jul 20 16:53:19.121020 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:19.120982 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k9tmt"] Jul 20 16:53:19.153085 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:19.153043 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rwdxc" event={"ID":"6e57b695-2dc8-436a-b18f-6236035f7214","Type":"ContainerStarted","Data":"89cbe8ab5608219dac1bf60b7645435c73ae390ee67df1f20584431a346cc95f"} Jul 20 16:53:19.200599 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:19.200554 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cksn2\" (UniqueName: \"kubernetes.io/projected/02bca006-6bd9-4eca-a684-49ce73cadbcd-kube-api-access-cksn2\") pod \"authorino-75d556bd77-k9tmt\" (UID: \"02bca006-6bd9-4eca-a684-49ce73cadbcd\") " pod="kuadrant-system/authorino-75d556bd77-k9tmt" Jul 20 16:53:19.302451 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:19.302080 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cksn2\" (UniqueName: \"kubernetes.io/projected/02bca006-6bd9-4eca-a684-49ce73cadbcd-kube-api-access-cksn2\") pod \"authorino-75d556bd77-k9tmt\" (UID: \"02bca006-6bd9-4eca-a684-49ce73cadbcd\") " pod="kuadrant-system/authorino-75d556bd77-k9tmt" Jul 20 16:53:19.313071 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:19.313033 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cksn2\" (UniqueName: \"kubernetes.io/projected/02bca006-6bd9-4eca-a684-49ce73cadbcd-kube-api-access-cksn2\") pod \"authorino-75d556bd77-k9tmt\" (UID: \"02bca006-6bd9-4eca-a684-49ce73cadbcd\") " pod="kuadrant-system/authorino-75d556bd77-k9tmt" Jul 20 16:53:19.425364 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:19.425267 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-k9tmt" Jul 20 16:53:19.605005 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:19.604949 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k9tmt"] Jul 20 16:53:19.607002 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:53:19.606971 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02bca006_6bd9_4eca_a684_49ce73cadbcd.slice/crio-aa9cecaf3c8cda1dee1eb58b7f198f19df8bd47b085c61d04e7fccdd2210f059 WatchSource:0}: Error finding container aa9cecaf3c8cda1dee1eb58b7f198f19df8bd47b085c61d04e7fccdd2210f059: Status 404 returned error can't find the container with id aa9cecaf3c8cda1dee1eb58b7f198f19df8bd47b085c61d04e7fccdd2210f059 Jul 20 16:53:20.159791 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:20.159731 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-k9tmt" event={"ID":"02bca006-6bd9-4eca-a684-49ce73cadbcd","Type":"ContainerStarted","Data":"aa9cecaf3c8cda1dee1eb58b7f198f19df8bd47b085c61d04e7fccdd2210f059"} Jul 20 16:53:22.179126 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:22.179084 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rwdxc" event={"ID":"6e57b695-2dc8-436a-b18f-6236035f7214","Type":"ContainerStarted","Data":"07b91df19faeb6da8fb0176bcfc97cce8c605c802ce80b56b2180573270ff4d0"} Jul 20 16:53:22.179507 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:22.179153 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-rwdxc" Jul 20 16:53:22.198655 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:22.198607 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-rwdxc" podStartSLOduration=2.381407321 podStartE2EDuration="5.19859061s" totalImagesPullingTime="2.817183289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:53:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:53:22.197401177 +0000 UTC m=+466.983400256" watchObservedRunningTime="2026-07-20 16:53:22.19859061 +0000 UTC m=+466.984589689" Jul 20 16:53:24.196665 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:24.196624 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-k9tmt" event={"ID":"02bca006-6bd9-4eca-a684-49ce73cadbcd","Type":"ContainerStarted","Data":"c0e78c1272d651129801a60497f3ef51a7662a03fe2578063c2fa86cc8156f79"} Jul 20 16:53:32.712625 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:32.712565 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-k9tmt" podStartSLOduration=10.164704294 podStartE2EDuration="13.712544921s" totalImagesPullingTime="3.547840627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:53:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:53:24.217655196 +0000 UTC m=+469.003654277" watchObservedRunningTime="2026-07-20 16:53:32.712544921 +0000 UTC m=+477.498543991" Jul 20 16:53:32.713138 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:32.712714 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rwdxc"] Jul 20 16:53:32.713138 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:32.713002 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-rwdxc" podUID="6e57b695-2dc8-436a-b18f-6236035f7214" containerName="limitador" containerID="cri-o://07b91df19faeb6da8fb0176bcfc97cce8c605c802ce80b56b2180573270ff4d0" gracePeriod=30 Jul 20 16:53:32.713655 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:32.713615 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-rwdxc" Jul 20 16:53:33.246429 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:33.246403 2566 generic.go:356] "Generic (PLEG): container finished" podID="6e57b695-2dc8-436a-b18f-6236035f7214" containerID="07b91df19faeb6da8fb0176bcfc97cce8c605c802ce80b56b2180573270ff4d0" exitCode=0 Jul 20 16:53:33.246485 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:33.246469 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rwdxc" event={"ID":"6e57b695-2dc8-436a-b18f-6236035f7214","Type":"ContainerDied","Data":"07b91df19faeb6da8fb0176bcfc97cce8c605c802ce80b56b2180573270ff4d0"} Jul 20 16:53:33.260925 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:33.260903 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rwdxc" Jul 20 16:53:33.336505 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:33.336476 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzl47\" (UniqueName: \"kubernetes.io/projected/6e57b695-2dc8-436a-b18f-6236035f7214-kube-api-access-nzl47\") pod \"6e57b695-2dc8-436a-b18f-6236035f7214\" (UID: \"6e57b695-2dc8-436a-b18f-6236035f7214\") " Jul 20 16:53:33.336686 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:33.336523 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6e57b695-2dc8-436a-b18f-6236035f7214-config-file\") pod \"6e57b695-2dc8-436a-b18f-6236035f7214\" (UID: \"6e57b695-2dc8-436a-b18f-6236035f7214\") " Jul 20 16:53:33.336928 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:33.336901 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e57b695-2dc8-436a-b18f-6236035f7214-config-file" (OuterVolumeSpecName: "config-file") pod "6e57b695-2dc8-436a-b18f-6236035f7214" (UID: "6e57b695-2dc8-436a-b18f-6236035f7214"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:53:33.338798 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:33.338758 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e57b695-2dc8-436a-b18f-6236035f7214-kube-api-access-nzl47" (OuterVolumeSpecName: "kube-api-access-nzl47") pod "6e57b695-2dc8-436a-b18f-6236035f7214" (UID: "6e57b695-2dc8-436a-b18f-6236035f7214"). InnerVolumeSpecName "kube-api-access-nzl47". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:53:33.437306 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:33.437269 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzl47\" (UniqueName: \"kubernetes.io/projected/6e57b695-2dc8-436a-b18f-6236035f7214-kube-api-access-nzl47\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:53:33.437306 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:33.437299 2566 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6e57b695-2dc8-436a-b18f-6236035f7214-config-file\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:53:34.252707 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.252675 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rwdxc" event={"ID":"6e57b695-2dc8-436a-b18f-6236035f7214","Type":"ContainerDied","Data":"89cbe8ab5608219dac1bf60b7645435c73ae390ee67df1f20584431a346cc95f"} Jul 20 16:53:34.253172 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.252715 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rwdxc" Jul 20 16:53:34.253172 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.252732 2566 scope.go:117] "RemoveContainer" containerID="07b91df19faeb6da8fb0176bcfc97cce8c605c802ce80b56b2180573270ff4d0" Jul 20 16:53:34.274260 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.274230 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rwdxc"] Jul 20 16:53:34.277074 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.277052 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rwdxc"] Jul 20 16:53:34.319217 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.319190 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7kmd4"] Jul 20 16:53:34.319567 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.319556 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e57b695-2dc8-436a-b18f-6236035f7214" containerName="limitador" Jul 20 16:53:34.319610 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.319569 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e57b695-2dc8-436a-b18f-6236035f7214" containerName="limitador" Jul 20 16:53:34.319645 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.319638 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e57b695-2dc8-436a-b18f-6236035f7214" containerName="limitador" Jul 20 16:53:34.323919 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.323898 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmd4" Jul 20 16:53:34.326084 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.326029 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 20 16:53:34.326214 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.326192 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-shs25\"" Jul 20 16:53:34.326451 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.326431 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 20 16:53:34.326559 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.326475 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 20 16:53:34.334078 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.334057 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7kmd4"] Jul 20 16:53:34.344805 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.344785 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0f7cddb2-d9a8-4822-bd10-cb86784a5a45-data\") pod \"postgres-69fb4594cf-7kmd4\" (UID: \"0f7cddb2-d9a8-4822-bd10-cb86784a5a45\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmd4" Jul 20 16:53:34.344922 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.344857 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n8gl\" (UniqueName: \"kubernetes.io/projected/0f7cddb2-d9a8-4822-bd10-cb86784a5a45-kube-api-access-2n8gl\") pod \"postgres-69fb4594cf-7kmd4\" (UID: \"0f7cddb2-d9a8-4822-bd10-cb86784a5a45\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmd4" Jul 20 16:53:34.445697 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.445655 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0f7cddb2-d9a8-4822-bd10-cb86784a5a45-data\") pod \"postgres-69fb4594cf-7kmd4\" (UID: \"0f7cddb2-d9a8-4822-bd10-cb86784a5a45\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmd4" Jul 20 16:53:34.445917 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.445711 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n8gl\" (UniqueName: \"kubernetes.io/projected/0f7cddb2-d9a8-4822-bd10-cb86784a5a45-kube-api-access-2n8gl\") pod \"postgres-69fb4594cf-7kmd4\" (UID: \"0f7cddb2-d9a8-4822-bd10-cb86784a5a45\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmd4" Jul 20 16:53:34.446073 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.446052 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0f7cddb2-d9a8-4822-bd10-cb86784a5a45-data\") pod \"postgres-69fb4594cf-7kmd4\" (UID: \"0f7cddb2-d9a8-4822-bd10-cb86784a5a45\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmd4" Jul 20 16:53:34.457336 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.457302 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n8gl\" (UniqueName: \"kubernetes.io/projected/0f7cddb2-d9a8-4822-bd10-cb86784a5a45-kube-api-access-2n8gl\") pod \"postgres-69fb4594cf-7kmd4\" (UID: \"0f7cddb2-d9a8-4822-bd10-cb86784a5a45\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmd4" Jul 20 16:53:34.636006 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.635969 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmd4" Jul 20 16:53:34.773704 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:34.773676 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7kmd4"] Jul 20 16:53:34.775408 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:53:34.775381 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f7cddb2_d9a8_4822_bd10_cb86784a5a45.slice/crio-677ce53f8e8b8986bb4c30d04bf6cfd92dcda2cbfbb400593b0b2123416c52aa WatchSource:0}: Error finding container 677ce53f8e8b8986bb4c30d04bf6cfd92dcda2cbfbb400593b0b2123416c52aa: Status 404 returned error can't find the container with id 677ce53f8e8b8986bb4c30d04bf6cfd92dcda2cbfbb400593b0b2123416c52aa Jul 20 16:53:35.259627 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:35.259593 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmd4" event={"ID":"0f7cddb2-d9a8-4822-bd10-cb86784a5a45","Type":"ContainerStarted","Data":"677ce53f8e8b8986bb4c30d04bf6cfd92dcda2cbfbb400593b0b2123416c52aa"} Jul 20 16:53:35.818731 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:35.818618 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e57b695-2dc8-436a-b18f-6236035f7214" path="/var/lib/kubelet/pods/6e57b695-2dc8-436a-b18f-6236035f7214/volumes" Jul 20 16:53:39.418926 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:39.418906 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 20 16:53:40.292041 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:40.291998 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmd4" event={"ID":"0f7cddb2-d9a8-4822-bd10-cb86784a5a45","Type":"ContainerStarted","Data":"32972af01495baa0595ac0cb702fd87555ae6f53c7f529ccaad775d7d4fe0e51"} Jul 20 16:53:40.292205 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:40.292122 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmd4" Jul 20 16:53:40.314331 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:40.314279 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmd4" podStartSLOduration=1.674402495 podStartE2EDuration="6.314263873s" totalImagesPullingTime="4.639861378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:53:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:53:40.311595977 +0000 UTC m=+485.097595057" watchObservedRunningTime="2026-07-20 16:53:40.314263873 +0000 UTC m=+485.100262979" Jul 20 16:53:46.327565 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:46.327532 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmd4" Jul 20 16:53:46.884132 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:46.884090 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kkh24"] Jul 20 16:53:46.890350 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:46.890320 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kkh24" Jul 20 16:53:46.891700 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:46.891675 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kkh24"] Jul 20 16:53:46.960138 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:46.960105 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prtxf\" (UniqueName: \"kubernetes.io/projected/2b838766-10d0-4290-a70e-a96903fa5630-kube-api-access-prtxf\") pod \"authorino-5d4b67c446-kkh24\" (UID: \"2b838766-10d0-4290-a70e-a96903fa5630\") " pod="kuadrant-system/authorino-5d4b67c446-kkh24" Jul 20 16:53:47.061323 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:47.061290 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prtxf\" (UniqueName: \"kubernetes.io/projected/2b838766-10d0-4290-a70e-a96903fa5630-kube-api-access-prtxf\") pod \"authorino-5d4b67c446-kkh24\" (UID: \"2b838766-10d0-4290-a70e-a96903fa5630\") " pod="kuadrant-system/authorino-5d4b67c446-kkh24" Jul 20 16:53:47.069855 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:47.069825 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prtxf\" (UniqueName: \"kubernetes.io/projected/2b838766-10d0-4290-a70e-a96903fa5630-kube-api-access-prtxf\") pod \"authorino-5d4b67c446-kkh24\" (UID: \"2b838766-10d0-4290-a70e-a96903fa5630\") " pod="kuadrant-system/authorino-5d4b67c446-kkh24" Jul 20 16:53:47.091692 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:47.091659 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kkh24"] Jul 20 16:53:47.091955 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:47.091942 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kkh24" Jul 20 16:53:47.120251 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:47.120222 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b79cfb9d4-sjm2r"] Jul 20 16:53:47.123365 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:47.123344 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b79cfb9d4-sjm2r" Jul 20 16:53:47.125534 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:47.125511 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 20 16:53:47.133506 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:47.133480 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b79cfb9d4-sjm2r"] Jul 20 16:53:47.236706 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:47.236664 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kkh24"] Jul 20 16:53:47.238962 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:53:47.238927 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b838766_10d0_4290_a70e_a96903fa5630.slice/crio-8d6a15786c386d01f74f645746c1dadac116de446a0c455e28363cef2343baa7 WatchSource:0}: Error finding container 8d6a15786c386d01f74f645746c1dadac116de446a0c455e28363cef2343baa7: Status 404 returned error can't find the container with id 8d6a15786c386d01f74f645746c1dadac116de446a0c455e28363cef2343baa7 Jul 20 16:53:47.263576 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:47.263547 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5631772d-cae2-4e79-909a-3c87935b0e9d-tls-cert\") pod \"authorino-b79cfb9d4-sjm2r\" (UID: \"5631772d-cae2-4e79-909a-3c87935b0e9d\") " pod="kuadrant-system/authorino-b79cfb9d4-sjm2r" Jul 20 16:53:47.263741 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:47.263583 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp92z\" (UniqueName: \"kubernetes.io/projected/5631772d-cae2-4e79-909a-3c87935b0e9d-kube-api-access-mp92z\") pod \"authorino-b79cfb9d4-sjm2r\" (UID: \"5631772d-cae2-4e79-909a-3c87935b0e9d\") " pod="kuadrant-system/authorino-b79cfb9d4-sjm2r" Jul 20 16:53:47.328933 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:47.328893 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-kkh24" event={"ID":"2b838766-10d0-4290-a70e-a96903fa5630","Type":"ContainerStarted","Data":"8d6a15786c386d01f74f645746c1dadac116de446a0c455e28363cef2343baa7"} Jul 20 16:53:47.364478 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:47.364442 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5631772d-cae2-4e79-909a-3c87935b0e9d-tls-cert\") pod \"authorino-b79cfb9d4-sjm2r\" (UID: \"5631772d-cae2-4e79-909a-3c87935b0e9d\") " pod="kuadrant-system/authorino-b79cfb9d4-sjm2r" Jul 20 16:53:47.364634 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:47.364484 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp92z\" (UniqueName: \"kubernetes.io/projected/5631772d-cae2-4e79-909a-3c87935b0e9d-kube-api-access-mp92z\") pod \"authorino-b79cfb9d4-sjm2r\" (UID: \"5631772d-cae2-4e79-909a-3c87935b0e9d\") " pod="kuadrant-system/authorino-b79cfb9d4-sjm2r" Jul 20 16:53:47.367197 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:47.367174 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5631772d-cae2-4e79-909a-3c87935b0e9d-tls-cert\") pod \"authorino-b79cfb9d4-sjm2r\" (UID: \"5631772d-cae2-4e79-909a-3c87935b0e9d\") " pod="kuadrant-system/authorino-b79cfb9d4-sjm2r" Jul 20 16:53:47.373170 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:47.373146 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp92z\" (UniqueName: \"kubernetes.io/projected/5631772d-cae2-4e79-909a-3c87935b0e9d-kube-api-access-mp92z\") pod \"authorino-b79cfb9d4-sjm2r\" (UID: \"5631772d-cae2-4e79-909a-3c87935b0e9d\") " pod="kuadrant-system/authorino-b79cfb9d4-sjm2r" Jul 20 16:53:47.434122 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:47.434019 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b79cfb9d4-sjm2r" Jul 20 16:53:47.571487 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:47.571462 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b79cfb9d4-sjm2r"] Jul 20 16:53:47.573454 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:53:47.573427 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5631772d_cae2_4e79_909a_3c87935b0e9d.slice/crio-8d6bae3a967457807050a376b7036e03b1ae51c5f1bbe4bcdea546cce861375d WatchSource:0}: Error finding container 8d6bae3a967457807050a376b7036e03b1ae51c5f1bbe4bcdea546cce861375d: Status 404 returned error can't find the container with id 8d6bae3a967457807050a376b7036e03b1ae51c5f1bbe4bcdea546cce861375d Jul 20 16:53:48.335152 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:48.335113 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-kkh24" event={"ID":"2b838766-10d0-4290-a70e-a96903fa5630","Type":"ContainerStarted","Data":"314789fdd0e419d1977d2e683f914054a770544d65aac61fda41764b8c3507c2"} Jul 20 16:53:48.335665 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:48.335140 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-kkh24" podUID="2b838766-10d0-4290-a70e-a96903fa5630" containerName="authorino" containerID="cri-o://314789fdd0e419d1977d2e683f914054a770544d65aac61fda41764b8c3507c2" gracePeriod=30 Jul 20 16:53:48.336794 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:48.336745 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b79cfb9d4-sjm2r" event={"ID":"5631772d-cae2-4e79-909a-3c87935b0e9d","Type":"ContainerStarted","Data":"a55166c8aa93b7c77b9cf10a0ed97a2513e42d2c609d8c3de0ca25108a45ed66"} Jul 20 16:53:48.336895 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:48.336808 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b79cfb9d4-sjm2r" event={"ID":"5631772d-cae2-4e79-909a-3c87935b0e9d","Type":"ContainerStarted","Data":"8d6bae3a967457807050a376b7036e03b1ae51c5f1bbe4bcdea546cce861375d"} Jul 20 16:53:48.351333 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:48.351287 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-kkh24" podStartSLOduration=2.060372917 podStartE2EDuration="2.351268611s" totalImagesPullingTime="290.895694ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:53:48.348950655 +0000 UTC m=+493.134949733" watchObservedRunningTime="2026-07-20 16:53:48.351268611 +0000 UTC m=+493.137267689" Jul 20 16:53:48.366473 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:48.366421 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b79cfb9d4-sjm2r" podStartSLOduration=1.050454058 podStartE2EDuration="1.366404773s" totalImagesPullingTime="315.950715ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:53:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:53:48.365293416 +0000 UTC m=+493.151292509" watchObservedRunningTime="2026-07-20 16:53:48.366404773 +0000 UTC m=+493.152403851" Jul 20 16:53:48.394475 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:48.394427 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k9tmt"] Jul 20 16:53:48.394656 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:48.394623 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-k9tmt" podUID="02bca006-6bd9-4eca-a684-49ce73cadbcd" containerName="authorino" containerID="cri-o://c0e78c1272d651129801a60497f3ef51a7662a03fe2578063c2fa86cc8156f79" gracePeriod=30 Jul 20 16:53:48.651229 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:48.651205 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kkh24" Jul 20 16:53:48.654336 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:48.654307 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-k9tmt" Jul 20 16:53:48.778567 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:48.778528 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cksn2\" (UniqueName: \"kubernetes.io/projected/02bca006-6bd9-4eca-a684-49ce73cadbcd-kube-api-access-cksn2\") pod \"02bca006-6bd9-4eca-a684-49ce73cadbcd\" (UID: \"02bca006-6bd9-4eca-a684-49ce73cadbcd\") " Jul 20 16:53:48.778567 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:48.778569 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prtxf\" (UniqueName: \"kubernetes.io/projected/2b838766-10d0-4290-a70e-a96903fa5630-kube-api-access-prtxf\") pod \"2b838766-10d0-4290-a70e-a96903fa5630\" (UID: \"2b838766-10d0-4290-a70e-a96903fa5630\") " Jul 20 16:53:48.780812 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:48.780751 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b838766-10d0-4290-a70e-a96903fa5630-kube-api-access-prtxf" (OuterVolumeSpecName: "kube-api-access-prtxf") pod "2b838766-10d0-4290-a70e-a96903fa5630" (UID: "2b838766-10d0-4290-a70e-a96903fa5630"). InnerVolumeSpecName "kube-api-access-prtxf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:53:48.780924 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:48.780840 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02bca006-6bd9-4eca-a684-49ce73cadbcd-kube-api-access-cksn2" (OuterVolumeSpecName: "kube-api-access-cksn2") pod "02bca006-6bd9-4eca-a684-49ce73cadbcd" (UID: "02bca006-6bd9-4eca-a684-49ce73cadbcd"). InnerVolumeSpecName "kube-api-access-cksn2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:53:48.879651 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:48.879560 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cksn2\" (UniqueName: \"kubernetes.io/projected/02bca006-6bd9-4eca-a684-49ce73cadbcd-kube-api-access-cksn2\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:53:48.879651 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:48.879590 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prtxf\" (UniqueName: \"kubernetes.io/projected/2b838766-10d0-4290-a70e-a96903fa5630-kube-api-access-prtxf\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:53:49.343007 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.342970 2566 generic.go:356] "Generic (PLEG): container finished" podID="2b838766-10d0-4290-a70e-a96903fa5630" containerID="314789fdd0e419d1977d2e683f914054a770544d65aac61fda41764b8c3507c2" exitCode=0 Jul 20 16:53:49.343494 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.343016 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kkh24" Jul 20 16:53:49.343494 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.343023 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-kkh24" event={"ID":"2b838766-10d0-4290-a70e-a96903fa5630","Type":"ContainerDied","Data":"314789fdd0e419d1977d2e683f914054a770544d65aac61fda41764b8c3507c2"} Jul 20 16:53:49.343494 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.343070 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-kkh24" event={"ID":"2b838766-10d0-4290-a70e-a96903fa5630","Type":"ContainerDied","Data":"8d6a15786c386d01f74f645746c1dadac116de446a0c455e28363cef2343baa7"} Jul 20 16:53:49.343494 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.343089 2566 scope.go:117] "RemoveContainer" containerID="314789fdd0e419d1977d2e683f914054a770544d65aac61fda41764b8c3507c2" Jul 20 16:53:49.344598 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.344573 2566 generic.go:356] "Generic (PLEG): container finished" podID="02bca006-6bd9-4eca-a684-49ce73cadbcd" containerID="c0e78c1272d651129801a60497f3ef51a7662a03fe2578063c2fa86cc8156f79" exitCode=0 Jul 20 16:53:49.344692 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.344644 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-k9tmt" Jul 20 16:53:49.344737 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.344717 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-k9tmt" event={"ID":"02bca006-6bd9-4eca-a684-49ce73cadbcd","Type":"ContainerDied","Data":"c0e78c1272d651129801a60497f3ef51a7662a03fe2578063c2fa86cc8156f79"} Jul 20 16:53:49.344811 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.344749 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-k9tmt" event={"ID":"02bca006-6bd9-4eca-a684-49ce73cadbcd","Type":"ContainerDied","Data":"aa9cecaf3c8cda1dee1eb58b7f198f19df8bd47b085c61d04e7fccdd2210f059"} Jul 20 16:53:49.352603 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.352587 2566 scope.go:117] "RemoveContainer" containerID="314789fdd0e419d1977d2e683f914054a770544d65aac61fda41764b8c3507c2" Jul 20 16:53:49.352927 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:53:49.352907 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"314789fdd0e419d1977d2e683f914054a770544d65aac61fda41764b8c3507c2\": container with ID starting with 314789fdd0e419d1977d2e683f914054a770544d65aac61fda41764b8c3507c2 not found: ID does not exist" containerID="314789fdd0e419d1977d2e683f914054a770544d65aac61fda41764b8c3507c2" Jul 20 16:53:49.353024 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.352936 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"314789fdd0e419d1977d2e683f914054a770544d65aac61fda41764b8c3507c2"} err="failed to get container status \"314789fdd0e419d1977d2e683f914054a770544d65aac61fda41764b8c3507c2\": rpc error: code = NotFound desc = could not find container \"314789fdd0e419d1977d2e683f914054a770544d65aac61fda41764b8c3507c2\": container with ID starting with 314789fdd0e419d1977d2e683f914054a770544d65aac61fda41764b8c3507c2 not found: ID does not exist" Jul 20 16:53:49.353024 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.352953 2566 scope.go:117] "RemoveContainer" containerID="c0e78c1272d651129801a60497f3ef51a7662a03fe2578063c2fa86cc8156f79" Jul 20 16:53:49.360996 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.360978 2566 scope.go:117] "RemoveContainer" containerID="c0e78c1272d651129801a60497f3ef51a7662a03fe2578063c2fa86cc8156f79" Jul 20 16:53:49.361247 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:53:49.361228 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0e78c1272d651129801a60497f3ef51a7662a03fe2578063c2fa86cc8156f79\": container with ID starting with c0e78c1272d651129801a60497f3ef51a7662a03fe2578063c2fa86cc8156f79 not found: ID does not exist" containerID="c0e78c1272d651129801a60497f3ef51a7662a03fe2578063c2fa86cc8156f79" Jul 20 16:53:49.361305 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.361255 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0e78c1272d651129801a60497f3ef51a7662a03fe2578063c2fa86cc8156f79"} err="failed to get container status \"c0e78c1272d651129801a60497f3ef51a7662a03fe2578063c2fa86cc8156f79\": rpc error: code = NotFound desc = could not find container \"c0e78c1272d651129801a60497f3ef51a7662a03fe2578063c2fa86cc8156f79\": container with ID starting with c0e78c1272d651129801a60497f3ef51a7662a03fe2578063c2fa86cc8156f79 not found: ID does not exist" Jul 20 16:53:49.366648 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.366626 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k9tmt"] Jul 20 16:53:49.370381 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.370361 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k9tmt"] Jul 20 16:53:49.383541 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.383516 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kkh24"] Jul 20 16:53:49.385375 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.385357 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kkh24"] Jul 20 16:53:49.816223 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.816140 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02bca006-6bd9-4eca-a684-49ce73cadbcd" path="/var/lib/kubelet/pods/02bca006-6bd9-4eca-a684-49ce73cadbcd/volumes" Jul 20 16:53:49.816468 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:53:49.816456 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b838766-10d0-4290-a70e-a96903fa5630" path="/var/lib/kubelet/pods/2b838766-10d0-4290-a70e-a96903fa5630/volumes" Jul 20 16:55:14.846058 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.846024 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s"] Jul 20 16:55:14.846518 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.846380 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02bca006-6bd9-4eca-a684-49ce73cadbcd" containerName="authorino" Jul 20 16:55:14.846518 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.846391 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="02bca006-6bd9-4eca-a684-49ce73cadbcd" containerName="authorino" Jul 20 16:55:14.846518 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.846414 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b838766-10d0-4290-a70e-a96903fa5630" containerName="authorino" Jul 20 16:55:14.846518 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.846419 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b838766-10d0-4290-a70e-a96903fa5630" containerName="authorino" Jul 20 16:55:14.846518 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.846476 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="02bca006-6bd9-4eca-a684-49ce73cadbcd" containerName="authorino" Jul 20 16:55:14.846518 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.846483 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b838766-10d0-4290-a70e-a96903fa5630" containerName="authorino" Jul 20 16:55:14.848470 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.848452 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:14.854617 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.854591 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 20 16:55:14.854746 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.854660 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k6shf\"" Jul 20 16:55:14.855471 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.855155 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 20 16:55:14.855471 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.855187 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 20 16:55:14.872178 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.872139 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s"] Jul 20 16:55:14.933820 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.933758 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc661ae8-5399-487b-ab29-84665fa72f41-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vf6s\" (UID: \"bc661ae8-5399-487b-ab29-84665fa72f41\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:14.933820 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.933822 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc661ae8-5399-487b-ab29-84665fa72f41-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vf6s\" (UID: \"bc661ae8-5399-487b-ab29-84665fa72f41\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:14.934080 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.933958 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc661ae8-5399-487b-ab29-84665fa72f41-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vf6s\" (UID: \"bc661ae8-5399-487b-ab29-84665fa72f41\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:14.934080 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.934010 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r7gg\" (UniqueName: \"kubernetes.io/projected/bc661ae8-5399-487b-ab29-84665fa72f41-kube-api-access-5r7gg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vf6s\" (UID: \"bc661ae8-5399-487b-ab29-84665fa72f41\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:14.934168 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.934081 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc661ae8-5399-487b-ab29-84665fa72f41-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vf6s\" (UID: \"bc661ae8-5399-487b-ab29-84665fa72f41\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:14.934168 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:14.934117 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc661ae8-5399-487b-ab29-84665fa72f41-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vf6s\" (UID: \"bc661ae8-5399-487b-ab29-84665fa72f41\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:15.035488 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:15.035448 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc661ae8-5399-487b-ab29-84665fa72f41-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vf6s\" (UID: \"bc661ae8-5399-487b-ab29-84665fa72f41\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:15.035702 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:15.035509 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r7gg\" (UniqueName: \"kubernetes.io/projected/bc661ae8-5399-487b-ab29-84665fa72f41-kube-api-access-5r7gg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vf6s\" (UID: \"bc661ae8-5399-487b-ab29-84665fa72f41\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:15.035702 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:15.035539 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc661ae8-5399-487b-ab29-84665fa72f41-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vf6s\" (UID: \"bc661ae8-5399-487b-ab29-84665fa72f41\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:15.035702 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:15.035574 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc661ae8-5399-487b-ab29-84665fa72f41-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vf6s\" (UID: \"bc661ae8-5399-487b-ab29-84665fa72f41\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:15.035916 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:15.035798 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc661ae8-5399-487b-ab29-84665fa72f41-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vf6s\" (UID: \"bc661ae8-5399-487b-ab29-84665fa72f41\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:15.035916 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:15.035862 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc661ae8-5399-487b-ab29-84665fa72f41-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vf6s\" (UID: \"bc661ae8-5399-487b-ab29-84665fa72f41\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:15.035916 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:15.035905 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc661ae8-5399-487b-ab29-84665fa72f41-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vf6s\" (UID: \"bc661ae8-5399-487b-ab29-84665fa72f41\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:15.036102 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:15.036085 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc661ae8-5399-487b-ab29-84665fa72f41-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vf6s\" (UID: \"bc661ae8-5399-487b-ab29-84665fa72f41\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:15.036206 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:15.036186 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc661ae8-5399-487b-ab29-84665fa72f41-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vf6s\" (UID: \"bc661ae8-5399-487b-ab29-84665fa72f41\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:15.038110 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:15.038088 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc661ae8-5399-487b-ab29-84665fa72f41-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vf6s\" (UID: \"bc661ae8-5399-487b-ab29-84665fa72f41\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:15.038213 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:15.038192 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc661ae8-5399-487b-ab29-84665fa72f41-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vf6s\" (UID: \"bc661ae8-5399-487b-ab29-84665fa72f41\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:15.044360 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:15.044334 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r7gg\" (UniqueName: \"kubernetes.io/projected/bc661ae8-5399-487b-ab29-84665fa72f41-kube-api-access-5r7gg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4vf6s\" (UID: \"bc661ae8-5399-487b-ab29-84665fa72f41\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:15.159213 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:15.159122 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:15.301735 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:15.301707 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s"] Jul 20 16:55:15.303977 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:55:15.303948 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc661ae8_5399_487b_ab29_84665fa72f41.slice/crio-65677afdf9f70c7973c73156bc39ada438baab3c6b40f23b006a89d68c9d7627 WatchSource:0}: Error finding container 65677afdf9f70c7973c73156bc39ada438baab3c6b40f23b006a89d68c9d7627: Status 404 returned error can't find the container with id 65677afdf9f70c7973c73156bc39ada438baab3c6b40f23b006a89d68c9d7627 Jul 20 16:55:15.822392 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:15.822345 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" event={"ID":"bc661ae8-5399-487b-ab29-84665fa72f41","Type":"ContainerStarted","Data":"65677afdf9f70c7973c73156bc39ada438baab3c6b40f23b006a89d68c9d7627"} Jul 20 16:55:16.829380 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:16.829345 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" event={"ID":"bc661ae8-5399-487b-ab29-84665fa72f41","Type":"ContainerStarted","Data":"d3e17fe94a2e09b49f6efe6986da964b49ab360983fe0f4fb9c60e6989220001"} Jul 20 16:55:16.855193 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:16.855135 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" podStartSLOduration=1.766946031 podStartE2EDuration="2.855117266s" totalImagesPullingTime="1.088171235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:55:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:55:16.85424786 +0000 UTC m=+581.640246938" watchObservedRunningTime="2026-07-20 16:55:16.855117266 +0000 UTC m=+581.641116344" Jul 20 16:55:25.159841 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:25.159768 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:25.160355 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:25.159870 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:25.172326 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:25.172296 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:25.888018 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:25.887991 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4vf6s" Jul 20 16:55:35.690887 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:35.690858 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 16:55:35.691610 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:35.691590 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 16:55:38.104158 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.104125 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24"] Jul 20 16:55:38.106561 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.106545 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.108659 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.108640 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 20 16:55:38.116086 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.116060 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24"] Jul 20 16:55:38.252006 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.251950 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxn99\" (UniqueName: \"kubernetes.io/projected/5699cd6e-e484-48ab-adb1-08339abca632-kube-api-access-dxn99\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nqs24\" (UID: \"5699cd6e-e484-48ab-adb1-08339abca632\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.252206 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.252069 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5699cd6e-e484-48ab-adb1-08339abca632-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nqs24\" (UID: \"5699cd6e-e484-48ab-adb1-08339abca632\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.252206 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.252101 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5699cd6e-e484-48ab-adb1-08339abca632-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nqs24\" (UID: \"5699cd6e-e484-48ab-adb1-08339abca632\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.252206 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.252126 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5699cd6e-e484-48ab-adb1-08339abca632-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nqs24\" (UID: \"5699cd6e-e484-48ab-adb1-08339abca632\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.252206 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.252172 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5699cd6e-e484-48ab-adb1-08339abca632-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nqs24\" (UID: \"5699cd6e-e484-48ab-adb1-08339abca632\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.252350 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.252206 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5699cd6e-e484-48ab-adb1-08339abca632-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nqs24\" (UID: \"5699cd6e-e484-48ab-adb1-08339abca632\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.352989 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.352947 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5699cd6e-e484-48ab-adb1-08339abca632-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nqs24\" (UID: \"5699cd6e-e484-48ab-adb1-08339abca632\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.352989 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.352990 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5699cd6e-e484-48ab-adb1-08339abca632-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nqs24\" (UID: \"5699cd6e-e484-48ab-adb1-08339abca632\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.353198 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.353019 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5699cd6e-e484-48ab-adb1-08339abca632-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nqs24\" (UID: \"5699cd6e-e484-48ab-adb1-08339abca632\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.353198 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.353045 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5699cd6e-e484-48ab-adb1-08339abca632-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nqs24\" (UID: \"5699cd6e-e484-48ab-adb1-08339abca632\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.353198 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.353124 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxn99\" (UniqueName: \"kubernetes.io/projected/5699cd6e-e484-48ab-adb1-08339abca632-kube-api-access-dxn99\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nqs24\" (UID: \"5699cd6e-e484-48ab-adb1-08339abca632\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.353322 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.353183 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5699cd6e-e484-48ab-adb1-08339abca632-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nqs24\" (UID: \"5699cd6e-e484-48ab-adb1-08339abca632\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.353408 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.353387 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5699cd6e-e484-48ab-adb1-08339abca632-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nqs24\" (UID: \"5699cd6e-e484-48ab-adb1-08339abca632\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.353456 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.353435 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5699cd6e-e484-48ab-adb1-08339abca632-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nqs24\" (UID: \"5699cd6e-e484-48ab-adb1-08339abca632\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.353501 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.353474 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5699cd6e-e484-48ab-adb1-08339abca632-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nqs24\" (UID: \"5699cd6e-e484-48ab-adb1-08339abca632\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.355628 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.355558 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5699cd6e-e484-48ab-adb1-08339abca632-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nqs24\" (UID: \"5699cd6e-e484-48ab-adb1-08339abca632\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.355763 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.355729 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5699cd6e-e484-48ab-adb1-08339abca632-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nqs24\" (UID: \"5699cd6e-e484-48ab-adb1-08339abca632\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.361017 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.360993 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxn99\" (UniqueName: \"kubernetes.io/projected/5699cd6e-e484-48ab-adb1-08339abca632-kube-api-access-dxn99\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nqs24\" (UID: \"5699cd6e-e484-48ab-adb1-08339abca632\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.418387 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.418345 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:38.557683 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.557657 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24"] Jul 20 16:55:38.559664 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:55:38.559637 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5699cd6e_e484_48ab_adb1_08339abca632.slice/crio-c9d01589b4813456183e316f58f79fc2827b811c6030ef283a77910be450a9a5 WatchSource:0}: Error finding container c9d01589b4813456183e316f58f79fc2827b811c6030ef283a77910be450a9a5: Status 404 returned error can't find the container with id c9d01589b4813456183e316f58f79fc2827b811c6030ef283a77910be450a9a5 Jul 20 16:55:38.561075 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.561056 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 16:55:38.945059 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.945010 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" event={"ID":"5699cd6e-e484-48ab-adb1-08339abca632","Type":"ContainerStarted","Data":"8af180bf26c82d1ce0490f030564deb09f2d9e21ab657ac7101f2ae7708ab225"} Jul 20 16:55:38.945236 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.945114 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" event={"ID":"5699cd6e-e484-48ab-adb1-08339abca632","Type":"ContainerStarted","Data":"c9d01589b4813456183e316f58f79fc2827b811c6030ef283a77910be450a9a5"} Jul 20 16:55:38.967603 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:38.967542 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" podStartSLOduration=0.805524974 podStartE2EDuration="967.521877ms" totalImagesPullingTime="161.996903ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:55:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:55:38.965271643 +0000 UTC m=+603.751270719" watchObservedRunningTime="2026-07-20 16:55:38.967521877 +0000 UTC m=+603.753520957" Jul 20 16:55:48.418812 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:48.418752 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:48.419257 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:48.418852 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:48.431863 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:48.431830 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:55:49.014164 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:55:49.014132 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nqs24" Jul 20 16:57:19.937396 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:19.937357 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7f87b86957-zw899"] Jul 20 16:57:19.941038 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:19.941014 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f87b86957-zw899" Jul 20 16:57:19.952893 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:19.952868 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7f87b86957-zw899"] Jul 20 16:57:19.978901 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:19.978873 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9391dfd4-e0c4-47c6-a12e-a25c97292cb9-tls-cert\") pod \"authorino-7f87b86957-zw899\" (UID: \"9391dfd4-e0c4-47c6-a12e-a25c97292cb9\") " pod="kuadrant-system/authorino-7f87b86957-zw899" Jul 20 16:57:19.979041 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:19.978906 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx2cq\" (UniqueName: \"kubernetes.io/projected/9391dfd4-e0c4-47c6-a12e-a25c97292cb9-kube-api-access-zx2cq\") pod \"authorino-7f87b86957-zw899\" (UID: \"9391dfd4-e0c4-47c6-a12e-a25c97292cb9\") " pod="kuadrant-system/authorino-7f87b86957-zw899" Jul 20 16:57:20.079720 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:20.079686 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9391dfd4-e0c4-47c6-a12e-a25c97292cb9-tls-cert\") pod \"authorino-7f87b86957-zw899\" (UID: \"9391dfd4-e0c4-47c6-a12e-a25c97292cb9\") " pod="kuadrant-system/authorino-7f87b86957-zw899" Jul 20 16:57:20.079720 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:20.079718 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx2cq\" (UniqueName: \"kubernetes.io/projected/9391dfd4-e0c4-47c6-a12e-a25c97292cb9-kube-api-access-zx2cq\") pod \"authorino-7f87b86957-zw899\" (UID: \"9391dfd4-e0c4-47c6-a12e-a25c97292cb9\") " pod="kuadrant-system/authorino-7f87b86957-zw899" Jul 20 16:57:20.082268 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:20.082245 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9391dfd4-e0c4-47c6-a12e-a25c97292cb9-tls-cert\") pod \"authorino-7f87b86957-zw899\" (UID: \"9391dfd4-e0c4-47c6-a12e-a25c97292cb9\") " pod="kuadrant-system/authorino-7f87b86957-zw899" Jul 20 16:57:20.087294 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:20.087275 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx2cq\" (UniqueName: \"kubernetes.io/projected/9391dfd4-e0c4-47c6-a12e-a25c97292cb9-kube-api-access-zx2cq\") pod \"authorino-7f87b86957-zw899\" (UID: \"9391dfd4-e0c4-47c6-a12e-a25c97292cb9\") " pod="kuadrant-system/authorino-7f87b86957-zw899" Jul 20 16:57:20.249638 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:20.249545 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f87b86957-zw899" Jul 20 16:57:20.389115 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:20.389089 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7f87b86957-zw899"] Jul 20 16:57:20.391016 ip-10-0-130-18 kubenswrapper[2566]: W0720 16:57:20.390989 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9391dfd4_e0c4_47c6_a12e_a25c97292cb9.slice/crio-363461ede5e2d35d3aa494b443209ba7d23b4867c56c60024ed3b0ab76d5d24a WatchSource:0}: Error finding container 363461ede5e2d35d3aa494b443209ba7d23b4867c56c60024ed3b0ab76d5d24a: Status 404 returned error can't find the container with id 363461ede5e2d35d3aa494b443209ba7d23b4867c56c60024ed3b0ab76d5d24a Jul 20 16:57:20.506074 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:20.505986 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f87b86957-zw899" event={"ID":"9391dfd4-e0c4-47c6-a12e-a25c97292cb9","Type":"ContainerStarted","Data":"363461ede5e2d35d3aa494b443209ba7d23b4867c56c60024ed3b0ab76d5d24a"} Jul 20 16:57:21.515070 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:21.515030 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f87b86957-zw899" event={"ID":"9391dfd4-e0c4-47c6-a12e-a25c97292cb9","Type":"ContainerStarted","Data":"2d758b3f1055ec1942cb027c42ab9d77e61a5014510511ca73bc86ad1f73b0cc"} Jul 20 16:57:21.540602 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:21.540533 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7f87b86957-zw899" podStartSLOduration=2.083611042 podStartE2EDuration="2.54050927s" totalImagesPullingTime="456.898228ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:57:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:57:21.538666756 +0000 UTC m=+706.324665836" watchObservedRunningTime="2026-07-20 16:57:21.54050927 +0000 UTC m=+706.326508350" Jul 20 16:57:21.585023 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:21.584982 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b79cfb9d4-sjm2r"] Jul 20 16:57:21.585469 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:21.585439 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b79cfb9d4-sjm2r" podUID="5631772d-cae2-4e79-909a-3c87935b0e9d" containerName="authorino" containerID="cri-o://a55166c8aa93b7c77b9cf10a0ed97a2513e42d2c609d8c3de0ca25108a45ed66" gracePeriod=30 Jul 20 16:57:21.853734 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:21.853703 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b79cfb9d4-sjm2r" Jul 20 16:57:21.900609 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:21.900574 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5631772d-cae2-4e79-909a-3c87935b0e9d-tls-cert\") pod \"5631772d-cae2-4e79-909a-3c87935b0e9d\" (UID: \"5631772d-cae2-4e79-909a-3c87935b0e9d\") " Jul 20 16:57:21.900809 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:21.900632 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mp92z\" (UniqueName: \"kubernetes.io/projected/5631772d-cae2-4e79-909a-3c87935b0e9d-kube-api-access-mp92z\") pod \"5631772d-cae2-4e79-909a-3c87935b0e9d\" (UID: \"5631772d-cae2-4e79-909a-3c87935b0e9d\") " Jul 20 16:57:21.903031 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:21.902994 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5631772d-cae2-4e79-909a-3c87935b0e9d-kube-api-access-mp92z" (OuterVolumeSpecName: "kube-api-access-mp92z") pod "5631772d-cae2-4e79-909a-3c87935b0e9d" (UID: "5631772d-cae2-4e79-909a-3c87935b0e9d"). InnerVolumeSpecName "kube-api-access-mp92z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:57:21.914049 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:21.914019 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5631772d-cae2-4e79-909a-3c87935b0e9d-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5631772d-cae2-4e79-909a-3c87935b0e9d" (UID: "5631772d-cae2-4e79-909a-3c87935b0e9d"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:57:22.001706 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:22.001671 2566 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5631772d-cae2-4e79-909a-3c87935b0e9d-tls-cert\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:57:22.001706 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:22.001703 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mp92z\" (UniqueName: \"kubernetes.io/projected/5631772d-cae2-4e79-909a-3c87935b0e9d-kube-api-access-mp92z\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 16:57:22.522165 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:22.522067 2566 generic.go:356] "Generic (PLEG): container finished" podID="5631772d-cae2-4e79-909a-3c87935b0e9d" containerID="a55166c8aa93b7c77b9cf10a0ed97a2513e42d2c609d8c3de0ca25108a45ed66" exitCode=0 Jul 20 16:57:22.522165 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:22.522110 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b79cfb9d4-sjm2r" Jul 20 16:57:22.522165 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:22.522150 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b79cfb9d4-sjm2r" event={"ID":"5631772d-cae2-4e79-909a-3c87935b0e9d","Type":"ContainerDied","Data":"a55166c8aa93b7c77b9cf10a0ed97a2513e42d2c609d8c3de0ca25108a45ed66"} Jul 20 16:57:22.522704 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:22.522195 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b79cfb9d4-sjm2r" event={"ID":"5631772d-cae2-4e79-909a-3c87935b0e9d","Type":"ContainerDied","Data":"8d6bae3a967457807050a376b7036e03b1ae51c5f1bbe4bcdea546cce861375d"} Jul 20 16:57:22.522704 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:22.522216 2566 scope.go:117] "RemoveContainer" containerID="a55166c8aa93b7c77b9cf10a0ed97a2513e42d2c609d8c3de0ca25108a45ed66" Jul 20 16:57:22.533411 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:22.533393 2566 scope.go:117] "RemoveContainer" containerID="a55166c8aa93b7c77b9cf10a0ed97a2513e42d2c609d8c3de0ca25108a45ed66" Jul 20 16:57:22.533722 ip-10-0-130-18 kubenswrapper[2566]: E0720 16:57:22.533702 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a55166c8aa93b7c77b9cf10a0ed97a2513e42d2c609d8c3de0ca25108a45ed66\": container with ID starting with a55166c8aa93b7c77b9cf10a0ed97a2513e42d2c609d8c3de0ca25108a45ed66 not found: ID does not exist" containerID="a55166c8aa93b7c77b9cf10a0ed97a2513e42d2c609d8c3de0ca25108a45ed66" Jul 20 16:57:22.533810 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:22.533736 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a55166c8aa93b7c77b9cf10a0ed97a2513e42d2c609d8c3de0ca25108a45ed66"} err="failed to get container status \"a55166c8aa93b7c77b9cf10a0ed97a2513e42d2c609d8c3de0ca25108a45ed66\": rpc error: code = NotFound desc = could not find container \"a55166c8aa93b7c77b9cf10a0ed97a2513e42d2c609d8c3de0ca25108a45ed66\": container with ID starting with a55166c8aa93b7c77b9cf10a0ed97a2513e42d2c609d8c3de0ca25108a45ed66 not found: ID does not exist" Jul 20 16:57:22.553406 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:22.553375 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b79cfb9d4-sjm2r"] Jul 20 16:57:22.567555 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:22.567521 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b79cfb9d4-sjm2r"] Jul 20 16:57:23.815239 ip-10-0-130-18 kubenswrapper[2566]: I0720 16:57:23.815207 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5631772d-cae2-4e79-909a-3c87935b0e9d" path="/var/lib/kubelet/pods/5631772d-cae2-4e79-909a-3c87935b0e9d/volumes" Jul 20 17:00:00.140719 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:00.140672 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742780-9dkm2"] Jul 20 17:00:00.141202 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:00.141100 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5631772d-cae2-4e79-909a-3c87935b0e9d" containerName="authorino" Jul 20 17:00:00.141202 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:00.141112 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5631772d-cae2-4e79-909a-3c87935b0e9d" containerName="authorino" Jul 20 17:00:00.141202 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:00.141176 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5631772d-cae2-4e79-909a-3c87935b0e9d" containerName="authorino" Jul 20 17:00:00.144439 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:00.144420 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742780-9dkm2" Jul 20 17:00:00.146499 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:00.146470 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5jc2q\"" Jul 20 17:00:00.149416 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:00.149384 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742780-9dkm2"] Jul 20 17:00:00.156833 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:00.156804 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742780-9dkm2" Jul 20 17:00:00.295470 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:00.295440 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742780-9dkm2"] Jul 20 17:00:00.297430 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:00:00.297400 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87d40f2f_c4a2_4b93_a215_59e1b5cc5809.slice/crio-c308f98b28b3293aa5b962486e964668bdcda65bdc0f6966a3db4bdaed27ae83 WatchSource:0}: Error finding container c308f98b28b3293aa5b962486e964668bdcda65bdc0f6966a3db4bdaed27ae83: Status 404 returned error can't find the container with id c308f98b28b3293aa5b962486e964668bdcda65bdc0f6966a3db4bdaed27ae83 Jul 20 17:00:00.380640 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:00.380598 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742780-9dkm2" event={"ID":"87d40f2f-c4a2-4b93-a215-59e1b5cc5809","Type":"ContainerStarted","Data":"c308f98b28b3293aa5b962486e964668bdcda65bdc0f6966a3db4bdaed27ae83"} Jul 20 17:00:06.417893 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:06.417863 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742780-9dkm2" event={"ID":"87d40f2f-c4a2-4b93-a215-59e1b5cc5809","Type":"ContainerStarted","Data":"7509e1e6776bf76a1c045987427613952bff1e6f98433219dec17d3906e14973"} Jul 20 17:00:06.440388 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:06.440322 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742780-9dkm2" podStartSLOduration=0.692688424 podStartE2EDuration="6.440303962s" totalImagesPullingTime="5.747615538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:00:06.43713791 +0000 UTC m=+871.223136988" watchObservedRunningTime="2026-07-20 17:00:06.440303962 +0000 UTC m=+871.226303041" Jul 20 17:00:07.425651 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:07.425565 2566 generic.go:356] "Generic (PLEG): container finished" podID="87d40f2f-c4a2-4b93-a215-59e1b5cc5809" containerID="7509e1e6776bf76a1c045987427613952bff1e6f98433219dec17d3906e14973" exitCode=0 Jul 20 17:00:07.425651 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:07.425609 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742780-9dkm2" event={"ID":"87d40f2f-c4a2-4b93-a215-59e1b5cc5809","Type":"ContainerDied","Data":"7509e1e6776bf76a1c045987427613952bff1e6f98433219dec17d3906e14973"} Jul 20 17:00:08.567138 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:08.567111 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742780-9dkm2" Jul 20 17:00:09.437613 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:09.437578 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742780-9dkm2" event={"ID":"87d40f2f-c4a2-4b93-a215-59e1b5cc5809","Type":"ContainerDied","Data":"c308f98b28b3293aa5b962486e964668bdcda65bdc0f6966a3db4bdaed27ae83"} Jul 20 17:00:09.437802 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:09.437628 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c308f98b28b3293aa5b962486e964668bdcda65bdc0f6966a3db4bdaed27ae83" Jul 20 17:00:09.437802 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:09.437590 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742780-9dkm2" Jul 20 17:00:35.730481 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:35.730452 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 17:00:35.730926 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:00:35.730634 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 17:05:35.766935 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:05:35.766907 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 17:05:35.767462 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:05:35.767429 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 17:10:35.810453 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:10:35.810418 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 17:10:35.814106 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:10:35.814078 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 17:12:51.816368 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:51.816340 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw"] Jul 20 17:12:51.818669 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:51.816687 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87d40f2f-c4a2-4b93-a215-59e1b5cc5809" containerName="cleanup" Jul 20 17:12:51.818669 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:51.816705 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="87d40f2f-c4a2-4b93-a215-59e1b5cc5809" containerName="cleanup" Jul 20 17:12:51.818669 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:51.816795 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="87d40f2f-c4a2-4b93-a215-59e1b5cc5809" containerName="cleanup" Jul 20 17:12:51.819602 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:51.819587 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw" Jul 20 17:12:51.821701 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:51.821682 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5jc2q\"" Jul 20 17:12:51.828829 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:51.828804 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw"] Jul 20 17:12:51.902475 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:51.902442 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eda72910-143c-4cfa-b9c0-f9e3ac862981-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw\" (UID: \"eda72910-143c-4cfa-b9c0-f9e3ac862981\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw" Jul 20 17:12:52.004008 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:52.003971 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eda72910-143c-4cfa-b9c0-f9e3ac862981-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw\" (UID: \"eda72910-143c-4cfa-b9c0-f9e3ac862981\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw" Jul 20 17:12:52.004539 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:52.004519 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eda72910-143c-4cfa-b9c0-f9e3ac862981-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw\" (UID: \"eda72910-143c-4cfa-b9c0-f9e3ac862981\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw" Jul 20 17:12:52.132236 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:52.132202 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw" Jul 20 17:12:52.270314 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:52.270289 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw"] Jul 20 17:12:52.272671 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:12:52.272634 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda72910_143c_4cfa_b9c0_f9e3ac862981.slice/crio-7da8f3a6d2700ffe8bdb4cc83b2b02ead649d56d626d0c5628a08cc94e294fe3 WatchSource:0}: Error finding container 7da8f3a6d2700ffe8bdb4cc83b2b02ead649d56d626d0c5628a08cc94e294fe3: Status 404 returned error can't find the container with id 7da8f3a6d2700ffe8bdb4cc83b2b02ead649d56d626d0c5628a08cc94e294fe3 Jul 20 17:12:52.619467 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:52.619432 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw" event={"ID":"eda72910-143c-4cfa-b9c0-f9e3ac862981","Type":"ContainerStarted","Data":"c35f953a8c1cb79138cc1648b25b7a7f02c111060c120d52209f9bb9c90f300b"} Jul 20 17:12:52.619645 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:52.619493 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw" event={"ID":"eda72910-143c-4cfa-b9c0-f9e3ac862981","Type":"ContainerStarted","Data":"7da8f3a6d2700ffe8bdb4cc83b2b02ead649d56d626d0c5628a08cc94e294fe3"} Jul 20 17:12:52.635659 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:52.635582 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw" podStartSLOduration=1.63556686 podStartE2EDuration="1.63556686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:12:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:12:52.633898143 +0000 UTC m=+1637.419897222" watchObservedRunningTime="2026-07-20 17:12:52.63556686 +0000 UTC m=+1637.421565965" Jul 20 17:12:53.626752 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:53.626714 2566 generic.go:356] "Generic (PLEG): container finished" podID="eda72910-143c-4cfa-b9c0-f9e3ac862981" containerID="c35f953a8c1cb79138cc1648b25b7a7f02c111060c120d52209f9bb9c90f300b" exitCode=0 Jul 20 17:12:53.627163 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:53.626801 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw" event={"ID":"eda72910-143c-4cfa-b9c0-f9e3ac862981","Type":"ContainerDied","Data":"c35f953a8c1cb79138cc1648b25b7a7f02c111060c120d52209f9bb9c90f300b"} Jul 20 17:12:54.759324 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:54.759301 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw" Jul 20 17:12:54.830225 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:54.830191 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eda72910-143c-4cfa-b9c0-f9e3ac862981-maas-api-service-ca\") pod \"eda72910-143c-4cfa-b9c0-f9e3ac862981\" (UID: \"eda72910-143c-4cfa-b9c0-f9e3ac862981\") " Jul 20 17:12:54.830560 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:54.830539 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eda72910-143c-4cfa-b9c0-f9e3ac862981-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eda72910-143c-4cfa-b9c0-f9e3ac862981" (UID: "eda72910-143c-4cfa-b9c0-f9e3ac862981"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:12:54.931255 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:54.931177 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eda72910-143c-4cfa-b9c0-f9e3ac862981-maas-api-service-ca\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:12:55.639187 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:55.639151 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw" event={"ID":"eda72910-143c-4cfa-b9c0-f9e3ac862981","Type":"ContainerDied","Data":"7da8f3a6d2700ffe8bdb4cc83b2b02ead649d56d626d0c5628a08cc94e294fe3"} Jul 20 17:12:55.639187 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:55.639180 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw" Jul 20 17:12:55.639400 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:12:55.639198 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7da8f3a6d2700ffe8bdb4cc83b2b02ead649d56d626d0c5628a08cc94e294fe3" Jul 20 17:13:06.909504 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:06.909454 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw"] Jul 20 17:13:06.915606 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:06.915577 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-82838e1d-gwkfw"] Jul 20 17:13:07.816131 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:07.816092 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eda72910-143c-4cfa-b9c0-f9e3ac862981" path="/var/lib/kubelet/pods/eda72910-143c-4cfa-b9c0-f9e3ac862981/volumes" Jul 20 17:13:08.318216 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.318178 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5"] Jul 20 17:13:08.318630 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.318567 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eda72910-143c-4cfa-b9c0-f9e3ac862981" containerName="revoke-keys" Jul 20 17:13:08.318630 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.318579 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="eda72910-143c-4cfa-b9c0-f9e3ac862981" containerName="revoke-keys" Jul 20 17:13:08.318717 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.318668 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="eda72910-143c-4cfa-b9c0-f9e3ac862981" containerName="revoke-keys" Jul 20 17:13:08.321895 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.321868 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.323989 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.323971 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 20 17:13:08.324341 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.324325 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-df6eeabb-gw-openshift-default-dockercfg-492dg\"" Jul 20 17:13:08.347050 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.347018 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5"] Jul 20 17:13:08.458070 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.458029 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh7bw\" (UniqueName: \"kubernetes.io/projected/f0319bcf-9f1d-421b-9e74-d2acdab88e38-kube-api-access-hh7bw\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.458250 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.458094 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-token\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.458250 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.458116 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-workload-certs\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.458250 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.458135 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-credential-socket\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.458250 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.458152 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-podinfo\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.458250 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.458220 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-data\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.458423 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.458284 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-workload-socket\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.458423 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.458311 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-envoy\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.458423 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.458387 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istiod-ca-cert\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.558987 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.558941 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-token\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.558987 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.558983 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-workload-certs\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.559235 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.559000 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-credential-socket\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.559235 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.559022 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-podinfo\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.559235 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.559055 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-data\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.559235 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.559098 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-workload-socket\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.559235 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.559136 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-envoy\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.559235 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.559185 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istiod-ca-cert\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.559235 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.559225 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh7bw\" (UniqueName: \"kubernetes.io/projected/f0319bcf-9f1d-421b-9e74-d2acdab88e38-kube-api-access-hh7bw\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.559597 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.559440 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-workload-certs\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.559597 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.559514 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-data\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.559597 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.559570 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-workload-socket\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.559768 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.559748 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-credential-socket\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.559934 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.559917 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istiod-ca-cert\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.561616 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.561590 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-envoy\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.561862 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.561844 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-podinfo\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.570175 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.570104 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-token\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.571053 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.571031 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh7bw\" (UniqueName: \"kubernetes.io/projected/f0319bcf-9f1d-421b-9e74-d2acdab88e38-kube-api-access-hh7bw\") pod \"e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.632171 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.632135 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:08.787479 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.787454 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5"] Jul 20 17:13:08.789788 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:13:08.789730 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0319bcf_9f1d_421b_9e74_d2acdab88e38.slice/crio-89242f7e88d18218bc6ba927f0eaa84b707d0ca4b75d95722b6ed32c3c8ea141 WatchSource:0}: Error finding container 89242f7e88d18218bc6ba927f0eaa84b707d0ca4b75d95722b6ed32c3c8ea141: Status 404 returned error can't find the container with id 89242f7e88d18218bc6ba927f0eaa84b707d0ca4b75d95722b6ed32c3c8ea141 Jul 20 17:13:08.793175 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:08.792544 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 17:13:09.717158 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:09.717119 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" event={"ID":"f0319bcf-9f1d-421b-9e74-d2acdab88e38","Type":"ContainerStarted","Data":"89242f7e88d18218bc6ba927f0eaa84b707d0ca4b75d95722b6ed32c3c8ea141"} Jul 20 17:13:11.732108 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:11.732068 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:13:11.732397 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:11.732144 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:13:11.732397 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:11.732172 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:13:12.735833 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:12.735798 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" event={"ID":"f0319bcf-9f1d-421b-9e74-d2acdab88e38","Type":"ContainerStarted","Data":"d2104933a1d8c2ea9a1b99d8a10ed275da6cb1aabbbd7049743ec1c9d7e7cd4b"} Jul 20 17:13:12.757210 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:12.757161 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" podStartSLOduration=1.8181250009999999 podStartE2EDuration="4.757145938s" totalImagesPullingTime="2.939020937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:13:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:13:12.753852907 +0000 UTC m=+1657.539851985" watchObservedRunningTime="2026-07-20 17:13:12.757145938 +0000 UTC m=+1657.543145016" Jul 20 17:13:13.633088 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:13.633048 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:13.638113 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:13.638078 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:13.741033 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:13.740995 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:13.742095 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:13.742077 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:25.806820 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:25.806754 2566 generic.go:356] "Generic (PLEG): container finished" podID="f0319bcf-9f1d-421b-9e74-d2acdab88e38" containerID="d2104933a1d8c2ea9a1b99d8a10ed275da6cb1aabbbd7049743ec1c9d7e7cd4b" exitCode=0 Jul 20 17:13:25.807233 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:25.806835 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" event={"ID":"f0319bcf-9f1d-421b-9e74-d2acdab88e38","Type":"ContainerDied","Data":"d2104933a1d8c2ea9a1b99d8a10ed275da6cb1aabbbd7049743ec1c9d7e7cd4b"} Jul 20 17:13:25.807282 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:25.807233 2566 scope.go:117] "RemoveContainer" containerID="d2104933a1d8c2ea9a1b99d8a10ed275da6cb1aabbbd7049743ec1c9d7e7cd4b" Jul 20 17:13:25.809236 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:25.809203 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:13:25.809339 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:25.809288 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:13:25.809339 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:25.809323 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:13:26.817553 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:26.817516 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" event={"ID":"f0319bcf-9f1d-421b-9e74-d2acdab88e38","Type":"ContainerStarted","Data":"baf8a8c38ce25cfd51e6fd6711d357edc4ad5fdf7f9f054cd3ba5804cc312386"} Jul 20 17:13:27.633896 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:27.633857 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:27.634321 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:27.634288 2566 patch_prober.go:28] interesting pod/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 20 17:13:27.634460 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:27.634358 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" podUID="f0319bcf-9f1d-421b-9e74-d2acdab88e38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 20 17:13:28.632756 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:28.632714 2566 patch_prober.go:28] interesting pod/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 20 17:13:28.633414 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:28.633358 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" podUID="f0319bcf-9f1d-421b-9e74-d2acdab88e38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 20 17:13:28.740924 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:28.740884 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:29.632691 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:29.632641 2566 patch_prober.go:28] interesting pod/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 20 17:13:29.632944 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:29.632726 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" podUID="f0319bcf-9f1d-421b-9e74-d2acdab88e38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 20 17:13:30.636366 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:30.636339 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:30.637331 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:30.637313 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:13:39.454105 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:39.454069 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-df6eeabb-7c2mp"] Jul 20 17:13:39.457567 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:39.457550 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-df6eeabb-7c2mp" Jul 20 17:13:39.459665 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:39.459648 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5jc2q\"" Jul 20 17:13:39.465705 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:39.465655 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-df6eeabb-7c2mp"] Jul 20 17:13:39.559187 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:39.559151 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ab74056-4f9e-4c79-b10c-2f8c0be50461-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-df6eeabb-7c2mp\" (UID: \"0ab74056-4f9e-4c79-b10c-2f8c0be50461\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-df6eeabb-7c2mp" Jul 20 17:13:39.660457 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:39.660418 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ab74056-4f9e-4c79-b10c-2f8c0be50461-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-df6eeabb-7c2mp\" (UID: \"0ab74056-4f9e-4c79-b10c-2f8c0be50461\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-df6eeabb-7c2mp" Jul 20 17:13:39.661111 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:39.661083 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ab74056-4f9e-4c79-b10c-2f8c0be50461-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-df6eeabb-7c2mp\" (UID: \"0ab74056-4f9e-4c79-b10c-2f8c0be50461\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-df6eeabb-7c2mp" Jul 20 17:13:39.769431 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:39.769337 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-df6eeabb-7c2mp" Jul 20 17:13:39.922223 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:39.922191 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-df6eeabb-7c2mp"] Jul 20 17:13:39.923571 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:13:39.923543 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ab74056_4f9e_4c79_b10c_2f8c0be50461.slice/crio-f0c52aebe5c3aad2ccdfd0109c586d6c238d4f4f22b8e7b0a83e680fd327aae5 WatchSource:0}: Error finding container f0c52aebe5c3aad2ccdfd0109c586d6c238d4f4f22b8e7b0a83e680fd327aae5: Status 404 returned error can't find the container with id f0c52aebe5c3aad2ccdfd0109c586d6c238d4f4f22b8e7b0a83e680fd327aae5 Jul 20 17:13:40.915570 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:40.915478 2566 generic.go:356] "Generic (PLEG): container finished" podID="0ab74056-4f9e-4c79-b10c-2f8c0be50461" containerID="d11784fd81327c0c26e36779c540ae69ffd1305134482c82333c28ef6b24117f" exitCode=0 Jul 20 17:13:40.915570 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:40.915555 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-df6eeabb-7c2mp" event={"ID":"0ab74056-4f9e-4c79-b10c-2f8c0be50461","Type":"ContainerDied","Data":"d11784fd81327c0c26e36779c540ae69ffd1305134482c82333c28ef6b24117f"} Jul 20 17:13:40.915993 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:40.915606 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-df6eeabb-7c2mp" event={"ID":"0ab74056-4f9e-4c79-b10c-2f8c0be50461","Type":"ContainerStarted","Data":"f0c52aebe5c3aad2ccdfd0109c586d6c238d4f4f22b8e7b0a83e680fd327aae5"} Jul 20 17:13:42.055559 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:42.055522 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-df6eeabb-7c2mp" Jul 20 17:13:42.183838 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:42.183739 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ab74056-4f9e-4c79-b10c-2f8c0be50461-maas-api-service-ca\") pod \"0ab74056-4f9e-4c79-b10c-2f8c0be50461\" (UID: \"0ab74056-4f9e-4c79-b10c-2f8c0be50461\") " Jul 20 17:13:42.184121 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:42.184096 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ab74056-4f9e-4c79-b10c-2f8c0be50461-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0ab74056-4f9e-4c79-b10c-2f8c0be50461" (UID: "0ab74056-4f9e-4c79-b10c-2f8c0be50461"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:13:42.284407 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:42.284373 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ab74056-4f9e-4c79-b10c-2f8c0be50461-maas-api-service-ca\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:13:42.928278 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:42.928246 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-df6eeabb-7c2mp" Jul 20 17:13:42.928451 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:42.928278 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-df6eeabb-7c2mp" event={"ID":"0ab74056-4f9e-4c79-b10c-2f8c0be50461","Type":"ContainerDied","Data":"f0c52aebe5c3aad2ccdfd0109c586d6c238d4f4f22b8e7b0a83e680fd327aae5"} Jul 20 17:13:42.928451 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:42.928324 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0c52aebe5c3aad2ccdfd0109c586d6c238d4f4f22b8e7b0a83e680fd327aae5" Jul 20 17:13:54.534299 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:54.534257 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-df6eeabb-7c2mp"] Jul 20 17:13:54.537338 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:54.537311 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-df6eeabb-7c2mp"] Jul 20 17:13:54.678688 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:54.678656 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5"] Jul 20 17:13:54.678912 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:54.678881 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" podUID="f0319bcf-9f1d-421b-9e74-d2acdab88e38" containerName="istio-proxy" containerID="cri-o://baf8a8c38ce25cfd51e6fd6711d357edc4ad5fdf7f9f054cd3ba5804cc312386" gracePeriod=30 Jul 20 17:13:55.816017 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:55.815980 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ab74056-4f9e-4c79-b10c-2f8c0be50461" path="/var/lib/kubelet/pods/0ab74056-4f9e-4c79-b10c-2f8c0be50461/volumes" Jul 20 17:13:59.939016 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:13:59.938952 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:14:00.026974 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.026938 2566 generic.go:356] "Generic (PLEG): container finished" podID="f0319bcf-9f1d-421b-9e74-d2acdab88e38" containerID="baf8a8c38ce25cfd51e6fd6711d357edc4ad5fdf7f9f054cd3ba5804cc312386" exitCode=0 Jul 20 17:14:00.027173 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.026988 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" event={"ID":"f0319bcf-9f1d-421b-9e74-d2acdab88e38","Type":"ContainerDied","Data":"baf8a8c38ce25cfd51e6fd6711d357edc4ad5fdf7f9f054cd3ba5804cc312386"} Jul 20 17:14:00.027173 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.027002 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" Jul 20 17:14:00.027173 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.027090 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5" event={"ID":"f0319bcf-9f1d-421b-9e74-d2acdab88e38","Type":"ContainerDied","Data":"89242f7e88d18218bc6ba927f0eaa84b707d0ca4b75d95722b6ed32c3c8ea141"} Jul 20 17:14:00.027173 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.027123 2566 scope.go:117] "RemoveContainer" containerID="baf8a8c38ce25cfd51e6fd6711d357edc4ad5fdf7f9f054cd3ba5804cc312386" Jul 20 17:14:00.036204 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.036187 2566 scope.go:117] "RemoveContainer" containerID="d2104933a1d8c2ea9a1b99d8a10ed275da6cb1aabbbd7049743ec1c9d7e7cd4b" Jul 20 17:14:00.044754 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.044734 2566 scope.go:117] "RemoveContainer" containerID="baf8a8c38ce25cfd51e6fd6711d357edc4ad5fdf7f9f054cd3ba5804cc312386" Jul 20 17:14:00.045081 ip-10-0-130-18 kubenswrapper[2566]: E0720 17:14:00.045061 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baf8a8c38ce25cfd51e6fd6711d357edc4ad5fdf7f9f054cd3ba5804cc312386\": container with ID starting with baf8a8c38ce25cfd51e6fd6711d357edc4ad5fdf7f9f054cd3ba5804cc312386 not found: ID does not exist" containerID="baf8a8c38ce25cfd51e6fd6711d357edc4ad5fdf7f9f054cd3ba5804cc312386" Jul 20 17:14:00.045142 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.045093 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baf8a8c38ce25cfd51e6fd6711d357edc4ad5fdf7f9f054cd3ba5804cc312386"} err="failed to get container status \"baf8a8c38ce25cfd51e6fd6711d357edc4ad5fdf7f9f054cd3ba5804cc312386\": rpc error: code = NotFound desc = could not find container \"baf8a8c38ce25cfd51e6fd6711d357edc4ad5fdf7f9f054cd3ba5804cc312386\": container with ID starting with baf8a8c38ce25cfd51e6fd6711d357edc4ad5fdf7f9f054cd3ba5804cc312386 not found: ID does not exist" Jul 20 17:14:00.045142 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.045114 2566 scope.go:117] "RemoveContainer" containerID="d2104933a1d8c2ea9a1b99d8a10ed275da6cb1aabbbd7049743ec1c9d7e7cd4b" Jul 20 17:14:00.045371 ip-10-0-130-18 kubenswrapper[2566]: E0720 17:14:00.045355 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2104933a1d8c2ea9a1b99d8a10ed275da6cb1aabbbd7049743ec1c9d7e7cd4b\": container with ID starting with d2104933a1d8c2ea9a1b99d8a10ed275da6cb1aabbbd7049743ec1c9d7e7cd4b not found: ID does not exist" containerID="d2104933a1d8c2ea9a1b99d8a10ed275da6cb1aabbbd7049743ec1c9d7e7cd4b" Jul 20 17:14:00.045427 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.045374 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2104933a1d8c2ea9a1b99d8a10ed275da6cb1aabbbd7049743ec1c9d7e7cd4b"} err="failed to get container status \"d2104933a1d8c2ea9a1b99d8a10ed275da6cb1aabbbd7049743ec1c9d7e7cd4b\": rpc error: code = NotFound desc = could not find container \"d2104933a1d8c2ea9a1b99d8a10ed275da6cb1aabbbd7049743ec1c9d7e7cd4b\": container with ID starting with d2104933a1d8c2ea9a1b99d8a10ed275da6cb1aabbbd7049743ec1c9d7e7cd4b not found: ID does not exist" Jul 20 17:14:00.052716 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.052698 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-data\") pod \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " Jul 20 17:14:00.052822 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.052764 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-workload-certs\") pod \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " Jul 20 17:14:00.052822 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.052815 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-workload-socket\") pod \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " Jul 20 17:14:00.052917 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.052837 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hh7bw\" (UniqueName: \"kubernetes.io/projected/f0319bcf-9f1d-421b-9e74-d2acdab88e38-kube-api-access-hh7bw\") pod \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " Jul 20 17:14:00.052917 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.052856 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-token\") pod \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " Jul 20 17:14:00.052917 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.052904 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istiod-ca-cert\") pod \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " Jul 20 17:14:00.053069 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.052933 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-envoy\") pod \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " Jul 20 17:14:00.053069 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.052970 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-credential-socket\") pod \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " Jul 20 17:14:00.053069 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.052993 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f0319bcf-9f1d-421b-9e74-d2acdab88e38" (UID: "f0319bcf-9f1d-421b-9e74-d2acdab88e38"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:14:00.053069 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.053006 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-podinfo\") pod \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\" (UID: \"f0319bcf-9f1d-421b-9e74-d2acdab88e38\") " Jul 20 17:14:00.053291 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.053120 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f0319bcf-9f1d-421b-9e74-d2acdab88e38" (UID: "f0319bcf-9f1d-421b-9e74-d2acdab88e38"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:14:00.053370 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.053349 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f0319bcf-9f1d-421b-9e74-d2acdab88e38" (UID: "f0319bcf-9f1d-421b-9e74-d2acdab88e38"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:14:00.053418 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.053374 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-workload-certs\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:14:00.053418 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.053397 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-workload-socket\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:14:00.053811 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.053655 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f0319bcf-9f1d-421b-9e74-d2acdab88e38" (UID: "f0319bcf-9f1d-421b-9e74-d2acdab88e38"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:14:00.053811 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.053631 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-data" (OuterVolumeSpecName: "istio-data") pod "f0319bcf-9f1d-421b-9e74-d2acdab88e38" (UID: "f0319bcf-9f1d-421b-9e74-d2acdab88e38"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:14:00.055343 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.055320 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-token" (OuterVolumeSpecName: "istio-token") pod "f0319bcf-9f1d-421b-9e74-d2acdab88e38" (UID: "f0319bcf-9f1d-421b-9e74-d2acdab88e38"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:14:00.055439 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.055365 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f0319bcf-9f1d-421b-9e74-d2acdab88e38" (UID: "f0319bcf-9f1d-421b-9e74-d2acdab88e38"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:14:00.055611 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.055596 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f0319bcf-9f1d-421b-9e74-d2acdab88e38" (UID: "f0319bcf-9f1d-421b-9e74-d2acdab88e38"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 17:14:00.055678 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.055621 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0319bcf-9f1d-421b-9e74-d2acdab88e38-kube-api-access-hh7bw" (OuterVolumeSpecName: "kube-api-access-hh7bw") pod "f0319bcf-9f1d-421b-9e74-d2acdab88e38" (UID: "f0319bcf-9f1d-421b-9e74-d2acdab88e38"). InnerVolumeSpecName "kube-api-access-hh7bw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:14:00.153887 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.153851 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hh7bw\" (UniqueName: \"kubernetes.io/projected/f0319bcf-9f1d-421b-9e74-d2acdab88e38-kube-api-access-hh7bw\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:14:00.153887 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.153886 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-token\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:14:00.153887 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.153895 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istiod-ca-cert\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:14:00.154136 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.153904 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-envoy\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:14:00.154136 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.153912 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-credential-socket\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:14:00.154136 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.153920 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-podinfo\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:14:00.154136 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.153927 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0319bcf-9f1d-421b-9e74-d2acdab88e38-istio-data\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:14:00.349360 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.349328 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5"] Jul 20 17:14:00.354569 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:00.354544 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-df6eeabb-gw-openshift-default-6797f49cc9-wzrc5"] Jul 20 17:14:01.814740 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:01.814706 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0319bcf-9f1d-421b-9e74-d2acdab88e38" path="/var/lib/kubelet/pods/f0319bcf-9f1d-421b-9e74-d2acdab88e38/volumes" Jul 20 17:14:23.510244 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:23.510208 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9002fce-q6nqb"] Jul 20 17:14:23.510849 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:23.510818 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0319bcf-9f1d-421b-9e74-d2acdab88e38" containerName="istio-proxy" Jul 20 17:14:23.510849 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:23.510844 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0319bcf-9f1d-421b-9e74-d2acdab88e38" containerName="istio-proxy" Jul 20 17:14:23.511046 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:23.510870 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0319bcf-9f1d-421b-9e74-d2acdab88e38" containerName="istio-proxy" Jul 20 17:14:23.511046 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:23.510880 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0319bcf-9f1d-421b-9e74-d2acdab88e38" containerName="istio-proxy" Jul 20 17:14:23.511046 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:23.510897 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ab74056-4f9e-4c79-b10c-2f8c0be50461" containerName="revoke-keys" Jul 20 17:14:23.511046 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:23.510906 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ab74056-4f9e-4c79-b10c-2f8c0be50461" containerName="revoke-keys" Jul 20 17:14:23.511046 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:23.510996 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0319bcf-9f1d-421b-9e74-d2acdab88e38" containerName="istio-proxy" Jul 20 17:14:23.511046 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:23.511004 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ab74056-4f9e-4c79-b10c-2f8c0be50461" containerName="revoke-keys" Jul 20 17:14:23.511046 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:23.511011 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0319bcf-9f1d-421b-9e74-d2acdab88e38" containerName="istio-proxy" Jul 20 17:14:23.516179 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:23.516154 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9002fce-q6nqb" Jul 20 17:14:23.520092 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:23.520064 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5jc2q\"" Jul 20 17:14:23.522155 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:23.522130 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9002fce-q6nqb"] Jul 20 17:14:23.553728 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:23.553694 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f510663-3e4c-41bc-81d4-da4fe85a12a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e9002fce-q6nqb\" (UID: \"0f510663-3e4c-41bc-81d4-da4fe85a12a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9002fce-q6nqb" Jul 20 17:14:23.655043 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:23.655007 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f510663-3e4c-41bc-81d4-da4fe85a12a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e9002fce-q6nqb\" (UID: \"0f510663-3e4c-41bc-81d4-da4fe85a12a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9002fce-q6nqb" Jul 20 17:14:23.655582 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:23.655563 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f510663-3e4c-41bc-81d4-da4fe85a12a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e9002fce-q6nqb\" (UID: \"0f510663-3e4c-41bc-81d4-da4fe85a12a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9002fce-q6nqb" Jul 20 17:14:23.831755 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:23.831733 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9002fce-q6nqb" Jul 20 17:14:23.964783 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:23.964743 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9002fce-q6nqb"] Jul 20 17:14:23.966244 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:14:23.966218 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f510663_3e4c_41bc_81d4_da4fe85a12a0.slice/crio-771230b1ad7285656caf50a83e666c168f7482f3a7ad34077c15280344404074 WatchSource:0}: Error finding container 771230b1ad7285656caf50a83e666c168f7482f3a7ad34077c15280344404074: Status 404 returned error can't find the container with id 771230b1ad7285656caf50a83e666c168f7482f3a7ad34077c15280344404074 Jul 20 17:14:24.163791 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:24.163742 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9002fce-q6nqb" event={"ID":"0f510663-3e4c-41bc-81d4-da4fe85a12a0","Type":"ContainerStarted","Data":"771230b1ad7285656caf50a83e666c168f7482f3a7ad34077c15280344404074"} Jul 20 17:14:25.172143 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:25.172109 2566 generic.go:356] "Generic (PLEG): container finished" podID="0f510663-3e4c-41bc-81d4-da4fe85a12a0" containerID="fe5bef7bf8f0f50489c9ffe549a4fe0bd46b33e9d1d7c2e2ddedce8631db2b7d" exitCode=0 Jul 20 17:14:25.172532 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:25.172190 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9002fce-q6nqb" event={"ID":"0f510663-3e4c-41bc-81d4-da4fe85a12a0","Type":"ContainerDied","Data":"fe5bef7bf8f0f50489c9ffe549a4fe0bd46b33e9d1d7c2e2ddedce8631db2b7d"} Jul 20 17:14:26.313285 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:26.313258 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9002fce-q6nqb" Jul 20 17:14:26.379398 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:26.379361 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f510663-3e4c-41bc-81d4-da4fe85a12a0-maas-api-service-ca\") pod \"0f510663-3e4c-41bc-81d4-da4fe85a12a0\" (UID: \"0f510663-3e4c-41bc-81d4-da4fe85a12a0\") " Jul 20 17:14:26.379738 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:26.379715 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f510663-3e4c-41bc-81d4-da4fe85a12a0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0f510663-3e4c-41bc-81d4-da4fe85a12a0" (UID: "0f510663-3e4c-41bc-81d4-da4fe85a12a0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:14:26.480180 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:26.480108 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f510663-3e4c-41bc-81d4-da4fe85a12a0-maas-api-service-ca\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:14:27.183931 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:27.183900 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9002fce-q6nqb" Jul 20 17:14:27.184103 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:27.183925 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9002fce-q6nqb" event={"ID":"0f510663-3e4c-41bc-81d4-da4fe85a12a0","Type":"ContainerDied","Data":"771230b1ad7285656caf50a83e666c168f7482f3a7ad34077c15280344404074"} Jul 20 17:14:27.184103 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:27.183975 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="771230b1ad7285656caf50a83e666c168f7482f3a7ad34077c15280344404074" Jul 20 17:14:38.601028 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:38.600947 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9002fce-q6nqb"] Jul 20 17:14:38.603445 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:38.603405 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9002fce-q6nqb"] Jul 20 17:14:39.823073 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:14:39.823042 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f510663-3e4c-41bc-81d4-da4fe85a12a0" path="/var/lib/kubelet/pods/0f510663-3e4c-41bc-81d4-da4fe85a12a0/volumes" Jul 20 17:15:00.141152 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:00.141112 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742795-n7nfv"] Jul 20 17:15:00.141577 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:00.141480 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f510663-3e4c-41bc-81d4-da4fe85a12a0" containerName="revoke-keys" Jul 20 17:15:00.141577 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:00.141491 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f510663-3e4c-41bc-81d4-da4fe85a12a0" containerName="revoke-keys" Jul 20 17:15:00.141577 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:00.141559 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f510663-3e4c-41bc-81d4-da4fe85a12a0" containerName="revoke-keys" Jul 20 17:15:00.144060 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:00.144044 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742795-n7nfv" Jul 20 17:15:00.146273 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:00.146253 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5jc2q\"" Jul 20 17:15:00.154588 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:00.154559 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742795-n7nfv" Jul 20 17:15:00.157129 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:00.157090 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-9185322f-29742795-n5qvv"] Jul 20 17:15:00.165396 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:00.165364 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742795-n7nfv"] Jul 20 17:15:00.165535 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:00.165506 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-9185322f-29742795-n5qvv" Jul 20 17:15:00.178137 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:00.178108 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-9185322f-29742795-n5qvv"] Jul 20 17:15:00.196500 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:00.196477 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-9185322f-29742795-n5qvv" Jul 20 17:15:00.323765 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:00.323723 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742795-n7nfv"] Jul 20 17:15:00.324997 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:15:00.324963 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce967b0d_a6e3_4019_aca6_515a738cb73f.slice/crio-0135284e10f18ce69dcbfcfcf61730523deddcb9c11fa663d31f3a0634a86fbb WatchSource:0}: Error finding container 0135284e10f18ce69dcbfcfcf61730523deddcb9c11fa663d31f3a0634a86fbb: Status 404 returned error can't find the container with id 0135284e10f18ce69dcbfcfcf61730523deddcb9c11fa663d31f3a0634a86fbb Jul 20 17:15:00.348340 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:00.348317 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-9185322f-29742795-n5qvv"] Jul 20 17:15:00.361033 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:15:00.361001 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63fbd7e2_8533_4452_ba37_a92c0b00afa9.slice/crio-8ec53660e0ea583a18f9a6a5d2c8d7da60d582b598241ed60890f70d287b14eb WatchSource:0}: Error finding container 8ec53660e0ea583a18f9a6a5d2c8d7da60d582b598241ed60890f70d287b14eb: Status 404 returned error can't find the container with id 8ec53660e0ea583a18f9a6a5d2c8d7da60d582b598241ed60890f70d287b14eb Jul 20 17:15:00.373343 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:00.373316 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-9185322f-29742795-n5qvv" event={"ID":"63fbd7e2-8533-4452-ba37-a92c0b00afa9","Type":"ContainerStarted","Data":"8ec53660e0ea583a18f9a6a5d2c8d7da60d582b598241ed60890f70d287b14eb"} Jul 20 17:15:00.374659 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:00.374632 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742795-n7nfv" event={"ID":"ce967b0d-a6e3-4019-aca6-515a738cb73f","Type":"ContainerStarted","Data":"0135284e10f18ce69dcbfcfcf61730523deddcb9c11fa663d31f3a0634a86fbb"} Jul 20 17:15:01.381633 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:01.381603 2566 generic.go:356] "Generic (PLEG): container finished" podID="ce967b0d-a6e3-4019-aca6-515a738cb73f" containerID="9fc5bd1fb7dc2c9522e55d38db99505e5459152114bba72f29ae1017255d07dd" exitCode=0 Jul 20 17:15:01.382066 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:01.381740 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742795-n7nfv" event={"ID":"ce967b0d-a6e3-4019-aca6-515a738cb73f","Type":"ContainerDied","Data":"9fc5bd1fb7dc2c9522e55d38db99505e5459152114bba72f29ae1017255d07dd"} Jul 20 17:15:01.383355 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:01.383333 2566 generic.go:356] "Generic (PLEG): container finished" podID="63fbd7e2-8533-4452-ba37-a92c0b00afa9" containerID="119ee4cc705d053d194694a38e3ae3ccfbaf6c8842c951a1b4f88c596041f392" exitCode=0 Jul 20 17:15:01.383462 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:01.383362 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-9185322f-29742795-n5qvv" event={"ID":"63fbd7e2-8533-4452-ba37-a92c0b00afa9","Type":"ContainerDied","Data":"119ee4cc705d053d194694a38e3ae3ccfbaf6c8842c951a1b4f88c596041f392"} Jul 20 17:15:02.532385 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:02.532357 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742795-n7nfv" Jul 20 17:15:02.535298 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:02.535280 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-9185322f-29742795-n5qvv" Jul 20 17:15:03.395584 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:03.395552 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-9185322f-29742795-n5qvv" Jul 20 17:15:03.395584 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:03.395561 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-9185322f-29742795-n5qvv" event={"ID":"63fbd7e2-8533-4452-ba37-a92c0b00afa9","Type":"ContainerDied","Data":"8ec53660e0ea583a18f9a6a5d2c8d7da60d582b598241ed60890f70d287b14eb"} Jul 20 17:15:03.395865 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:03.395610 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ec53660e0ea583a18f9a6a5d2c8d7da60d582b598241ed60890f70d287b14eb" Jul 20 17:15:03.396928 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:03.396909 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742795-n7nfv" Jul 20 17:15:03.397031 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:03.396941 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742795-n7nfv" event={"ID":"ce967b0d-a6e3-4019-aca6-515a738cb73f","Type":"ContainerDied","Data":"0135284e10f18ce69dcbfcfcf61730523deddcb9c11fa663d31f3a0634a86fbb"} Jul 20 17:15:03.397031 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:03.396976 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0135284e10f18ce69dcbfcfcf61730523deddcb9c11fa663d31f3a0634a86fbb" Jul 20 17:15:07.746346 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:07.746310 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9185322f-8p8ll"] Jul 20 17:15:07.746859 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:07.746680 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63fbd7e2-8533-4452-ba37-a92c0b00afa9" containerName="cleanup" Jul 20 17:15:07.746859 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:07.746693 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="63fbd7e2-8533-4452-ba37-a92c0b00afa9" containerName="cleanup" Jul 20 17:15:07.746859 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:07.746715 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce967b0d-a6e3-4019-aca6-515a738cb73f" containerName="cleanup" Jul 20 17:15:07.746859 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:07.746723 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce967b0d-a6e3-4019-aca6-515a738cb73f" containerName="cleanup" Jul 20 17:15:07.746859 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:07.746812 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce967b0d-a6e3-4019-aca6-515a738cb73f" containerName="cleanup" Jul 20 17:15:07.746859 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:07.746823 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="63fbd7e2-8533-4452-ba37-a92c0b00afa9" containerName="cleanup" Jul 20 17:15:07.749949 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:07.749933 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9185322f-8p8ll" Jul 20 17:15:07.752202 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:07.752181 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5jc2q\"" Jul 20 17:15:07.761010 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:07.760983 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9185322f-8p8ll"] Jul 20 17:15:07.847002 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:07.846970 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9185322f-8p8ll\" (UID: \"d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9185322f-8p8ll" Jul 20 17:15:07.948070 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:07.948031 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9185322f-8p8ll\" (UID: \"d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9185322f-8p8ll" Jul 20 17:15:07.948644 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:07.948621 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9185322f-8p8ll\" (UID: \"d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9185322f-8p8ll" Jul 20 17:15:08.060097 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:08.059997 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9185322f-8p8ll" Jul 20 17:15:08.205822 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:08.205761 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9185322f-8p8ll"] Jul 20 17:15:08.208599 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:15:08.208574 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8b9ab21_c63b_4a8a_bb83_c51d7bd07c77.slice/crio-592f6ed51bc64a009876cefb4748766db4c29138cba47c9e2ed35a40e4d2c03e WatchSource:0}: Error finding container 592f6ed51bc64a009876cefb4748766db4c29138cba47c9e2ed35a40e4d2c03e: Status 404 returned error can't find the container with id 592f6ed51bc64a009876cefb4748766db4c29138cba47c9e2ed35a40e4d2c03e Jul 20 17:15:08.426802 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:08.426757 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9185322f-8p8ll" event={"ID":"d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77","Type":"ContainerStarted","Data":"592f6ed51bc64a009876cefb4748766db4c29138cba47c9e2ed35a40e4d2c03e"} Jul 20 17:15:09.434047 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:09.434012 2566 generic.go:356] "Generic (PLEG): container finished" podID="d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77" containerID="1aa10104373b1d3b5475db729d19dbf8701009bde1ae9011516e11b459669706" exitCode=0 Jul 20 17:15:09.434426 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:09.434102 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9185322f-8p8ll" event={"ID":"d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77","Type":"ContainerDied","Data":"1aa10104373b1d3b5475db729d19dbf8701009bde1ae9011516e11b459669706"} Jul 20 17:15:10.577455 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:10.577429 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9185322f-8p8ll" Jul 20 17:15:10.674091 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:10.674057 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77-maas-api-service-ca\") pod \"d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77\" (UID: \"d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77\") " Jul 20 17:15:10.674415 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:10.674393 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77" (UID: "d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:15:10.775379 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:10.775299 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77-maas-api-service-ca\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:15:11.446038 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:11.446012 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9185322f-8p8ll" Jul 20 17:15:11.446215 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:11.446054 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9185322f-8p8ll" event={"ID":"d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77","Type":"ContainerDied","Data":"592f6ed51bc64a009876cefb4748766db4c29138cba47c9e2ed35a40e4d2c03e"} Jul 20 17:15:11.446215 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:11.446105 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="592f6ed51bc64a009876cefb4748766db4c29138cba47c9e2ed35a40e4d2c03e" Jul 20 17:15:18.331190 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:18.331147 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-9185322f-29742795-n5qvv"] Jul 20 17:15:18.334071 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:18.334041 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-9185322f-29742795-n5qvv"] Jul 20 17:15:19.815154 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:19.815118 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63fbd7e2-8533-4452-ba37-a92c0b00afa9" path="/var/lib/kubelet/pods/63fbd7e2-8533-4452-ba37-a92c0b00afa9/volumes" Jul 20 17:15:22.854478 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:22.854432 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9185322f-8p8ll"] Jul 20 17:15:22.858793 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:22.858749 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9185322f-8p8ll"] Jul 20 17:15:23.821249 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:23.816054 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77" path="/var/lib/kubelet/pods/d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77/volumes" Jul 20 17:15:35.859077 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:35.859048 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 17:15:35.865202 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:15:35.865183 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 17:17:23.333081 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:23.333046 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq"] Jul 20 17:17:23.333683 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:23.333421 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77" containerName="revoke-keys" Jul 20 17:17:23.333683 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:23.333432 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77" containerName="revoke-keys" Jul 20 17:17:23.333683 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:23.333501 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8b9ab21-c63b-4a8a-bb83-c51d7bd07c77" containerName="revoke-keys" Jul 20 17:17:23.336535 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:23.336516 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq" Jul 20 17:17:23.339251 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:23.339225 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5jc2q\"" Jul 20 17:17:23.399748 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:23.399708 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq"] Jul 20 17:17:23.400185 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:23.400141 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05234c8e-ea32-45b9-9359-ff94d66f4553-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq\" (UID: \"05234c8e-ea32-45b9-9359-ff94d66f4553\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq" Jul 20 17:17:23.501239 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:23.501199 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05234c8e-ea32-45b9-9359-ff94d66f4553-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq\" (UID: \"05234c8e-ea32-45b9-9359-ff94d66f4553\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq" Jul 20 17:17:23.501844 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:23.501826 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05234c8e-ea32-45b9-9359-ff94d66f4553-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq\" (UID: \"05234c8e-ea32-45b9-9359-ff94d66f4553\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq" Jul 20 17:17:23.647323 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:23.647231 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq" Jul 20 17:17:23.792631 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:23.792605 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq"] Jul 20 17:17:23.794714 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:17:23.794682 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05234c8e_ea32_45b9_9359_ff94d66f4553.slice/crio-89cba7770183fd54712346a8193eb6affab66d4c5ec1501ff175332b0f9fedae WatchSource:0}: Error finding container 89cba7770183fd54712346a8193eb6affab66d4c5ec1501ff175332b0f9fedae: Status 404 returned error can't find the container with id 89cba7770183fd54712346a8193eb6affab66d4c5ec1501ff175332b0f9fedae Jul 20 17:17:24.199389 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:24.199294 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq" event={"ID":"05234c8e-ea32-45b9-9359-ff94d66f4553","Type":"ContainerStarted","Data":"0e85d409cac138be3869f8241b4a91967dbb637e8aa56cefd00549217e345bdf"} Jul 20 17:17:24.199389 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:24.199352 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq" event={"ID":"05234c8e-ea32-45b9-9359-ff94d66f4553","Type":"ContainerStarted","Data":"89cba7770183fd54712346a8193eb6affab66d4c5ec1501ff175332b0f9fedae"} Jul 20 17:17:24.220516 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:24.220454 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq" podStartSLOduration=1.220439153 podStartE2EDuration="1.220439153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:17:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:17:24.219462221 +0000 UTC m=+1909.005461300" watchObservedRunningTime="2026-07-20 17:17:24.220439153 +0000 UTC m=+1909.006438229" Jul 20 17:17:25.206948 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:25.206911 2566 generic.go:356] "Generic (PLEG): container finished" podID="05234c8e-ea32-45b9-9359-ff94d66f4553" containerID="0e85d409cac138be3869f8241b4a91967dbb637e8aa56cefd00549217e345bdf" exitCode=0 Jul 20 17:17:25.207326 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:25.206999 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq" event={"ID":"05234c8e-ea32-45b9-9359-ff94d66f4553","Type":"ContainerDied","Data":"0e85d409cac138be3869f8241b4a91967dbb637e8aa56cefd00549217e345bdf"} Jul 20 17:17:26.352454 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:26.352431 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq" Jul 20 17:17:26.431238 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:26.431206 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05234c8e-ea32-45b9-9359-ff94d66f4553-maas-api-service-ca\") pod \"05234c8e-ea32-45b9-9359-ff94d66f4553\" (UID: \"05234c8e-ea32-45b9-9359-ff94d66f4553\") " Jul 20 17:17:26.431587 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:26.431561 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05234c8e-ea32-45b9-9359-ff94d66f4553-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "05234c8e-ea32-45b9-9359-ff94d66f4553" (UID: "05234c8e-ea32-45b9-9359-ff94d66f4553"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:17:26.532406 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:26.532328 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05234c8e-ea32-45b9-9359-ff94d66f4553-maas-api-service-ca\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:17:27.218998 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:27.218968 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq" Jul 20 17:17:27.218998 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:27.218989 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq" event={"ID":"05234c8e-ea32-45b9-9359-ff94d66f4553","Type":"ContainerDied","Data":"89cba7770183fd54712346a8193eb6affab66d4c5ec1501ff175332b0f9fedae"} Jul 20 17:17:27.219216 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:27.219037 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89cba7770183fd54712346a8193eb6affab66d4c5ec1501ff175332b0f9fedae" Jul 20 17:17:38.447760 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:38.447724 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq"] Jul 20 17:17:38.450865 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:38.450840 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fd39db2f-qp7jq"] Jul 20 17:17:39.815135 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:17:39.815101 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05234c8e-ea32-45b9-9359-ff94d66f4553" path="/var/lib/kubelet/pods/05234c8e-ea32-45b9-9359-ff94d66f4553/volumes" Jul 20 17:18:45.710696 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:45.710656 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn"] Jul 20 17:18:45.711305 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:45.711244 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05234c8e-ea32-45b9-9359-ff94d66f4553" containerName="revoke-keys" Jul 20 17:18:45.711305 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:45.711265 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="05234c8e-ea32-45b9-9359-ff94d66f4553" containerName="revoke-keys" Jul 20 17:18:45.711430 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:45.711374 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="05234c8e-ea32-45b9-9359-ff94d66f4553" containerName="revoke-keys" Jul 20 17:18:45.714642 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:45.714620 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn" Jul 20 17:18:45.717017 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:45.716882 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5jc2q\"" Jul 20 17:18:45.720612 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:45.720577 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn"] Jul 20 17:18:45.847025 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:45.846996 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2114ce3-b016-46f8-bb87-386a7dcd81f2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn\" (UID: \"c2114ce3-b016-46f8-bb87-386a7dcd81f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn" Jul 20 17:18:45.948153 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:45.948122 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2114ce3-b016-46f8-bb87-386a7dcd81f2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn\" (UID: \"c2114ce3-b016-46f8-bb87-386a7dcd81f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn" Jul 20 17:18:45.948731 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:45.948713 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2114ce3-b016-46f8-bb87-386a7dcd81f2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn\" (UID: \"c2114ce3-b016-46f8-bb87-386a7dcd81f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn" Jul 20 17:18:46.027512 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:46.027425 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn" Jul 20 17:18:46.158520 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:46.158497 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn"] Jul 20 17:18:46.160979 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:18:46.160949 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2114ce3_b016_46f8_bb87_386a7dcd81f2.slice/crio-332ab7bd7fe86f47a8fd0a9f334c384bd58cd921eb028b56bbad9522367e0afd WatchSource:0}: Error finding container 332ab7bd7fe86f47a8fd0a9f334c384bd58cd921eb028b56bbad9522367e0afd: Status 404 returned error can't find the container with id 332ab7bd7fe86f47a8fd0a9f334c384bd58cd921eb028b56bbad9522367e0afd Jul 20 17:18:46.669817 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:46.669766 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn" event={"ID":"c2114ce3-b016-46f8-bb87-386a7dcd81f2","Type":"ContainerStarted","Data":"f02a7678da086885914cfbabf7a36b993f6315e882dfe8281bb59322628fcebb"} Jul 20 17:18:46.669978 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:46.669842 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn" event={"ID":"c2114ce3-b016-46f8-bb87-386a7dcd81f2","Type":"ContainerStarted","Data":"332ab7bd7fe86f47a8fd0a9f334c384bd58cd921eb028b56bbad9522367e0afd"} Jul 20 17:18:46.685300 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:46.685252 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn" podStartSLOduration=1.685238473 podStartE2EDuration="1.685238473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:18:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:18:46.683371191 +0000 UTC m=+1991.469370283" watchObservedRunningTime="2026-07-20 17:18:46.685238473 +0000 UTC m=+1991.471237600" Jul 20 17:18:47.677389 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:47.677356 2566 generic.go:356] "Generic (PLEG): container finished" podID="c2114ce3-b016-46f8-bb87-386a7dcd81f2" containerID="f02a7678da086885914cfbabf7a36b993f6315e882dfe8281bb59322628fcebb" exitCode=0 Jul 20 17:18:47.677756 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:47.677447 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn" event={"ID":"c2114ce3-b016-46f8-bb87-386a7dcd81f2","Type":"ContainerDied","Data":"f02a7678da086885914cfbabf7a36b993f6315e882dfe8281bb59322628fcebb"} Jul 20 17:18:48.812010 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:48.811985 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn" Jul 20 17:18:48.976748 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:48.976647 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2114ce3-b016-46f8-bb87-386a7dcd81f2-maas-api-service-ca\") pod \"c2114ce3-b016-46f8-bb87-386a7dcd81f2\" (UID: \"c2114ce3-b016-46f8-bb87-386a7dcd81f2\") " Jul 20 17:18:48.977031 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:48.977004 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2114ce3-b016-46f8-bb87-386a7dcd81f2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c2114ce3-b016-46f8-bb87-386a7dcd81f2" (UID: "c2114ce3-b016-46f8-bb87-386a7dcd81f2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:18:49.077389 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:49.077356 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2114ce3-b016-46f8-bb87-386a7dcd81f2-maas-api-service-ca\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:18:49.689277 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:49.689248 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn" Jul 20 17:18:49.689277 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:49.689264 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn" event={"ID":"c2114ce3-b016-46f8-bb87-386a7dcd81f2","Type":"ContainerDied","Data":"332ab7bd7fe86f47a8fd0a9f334c384bd58cd921eb028b56bbad9522367e0afd"} Jul 20 17:18:49.689499 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:18:49.689312 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="332ab7bd7fe86f47a8fd0a9f334c384bd58cd921eb028b56bbad9522367e0afd" Jul 20 17:19:05.827736 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:19:05.827700 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn"] Jul 20 17:19:05.831862 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:19:05.831833 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fe047e55-t8zhn"] Jul 20 17:19:07.815333 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:19:07.815294 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2114ce3-b016-46f8-bb87-386a7dcd81f2" path="/var/lib/kubelet/pods/c2114ce3-b016-46f8-bb87-386a7dcd81f2/volumes" Jul 20 17:19:35.947049 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:19:35.947017 2566 scope.go:117] "RemoveContainer" containerID="c35f953a8c1cb79138cc1648b25b7a7f02c111060c120d52209f9bb9c90f300b" Jul 20 17:20:35.898920 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:35.898894 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 17:20:35.905374 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:35.905353 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 17:20:35.969220 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:35.969188 2566 scope.go:117] "RemoveContainer" containerID="fe5bef7bf8f0f50489c9ffe549a4fe0bd46b33e9d1d7c2e2ddedce8631db2b7d" Jul 20 17:20:35.978072 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:35.978051 2566 scope.go:117] "RemoveContainer" containerID="d11784fd81327c0c26e36779c540ae69ffd1305134482c82333c28ef6b24117f" Jul 20 17:20:59.444742 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.444705 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42"] Jul 20 17:20:59.445276 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.445252 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2114ce3-b016-46f8-bb87-386a7dcd81f2" containerName="revoke-keys" Jul 20 17:20:59.445276 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.445268 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2114ce3-b016-46f8-bb87-386a7dcd81f2" containerName="revoke-keys" Jul 20 17:20:59.445399 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.445341 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2114ce3-b016-46f8-bb87-386a7dcd81f2" containerName="revoke-keys" Jul 20 17:20:59.448450 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.448429 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.452721 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.452689 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-0062e2-openshift-default-dockercfg-z22bv\"" Jul 20 17:20:59.452874 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.452750 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 20 17:20:59.466945 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.466915 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42"] Jul 20 17:20:59.569576 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.569538 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-istio-data\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.569576 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.569575 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w87w\" (UniqueName: \"kubernetes.io/projected/31b727dc-4050-4de6-ba38-88487799bda2-kube-api-access-2w87w\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.569834 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.569597 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-credential-socket\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.569834 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.569739 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-istio-envoy\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.569834 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.569804 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31b727dc-4050-4de6-ba38-88487799bda2-istio-token\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.569834 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.569825 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-workload-socket\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.569991 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.569840 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31b727dc-4050-4de6-ba38-88487799bda2-istio-podinfo\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.569991 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.569861 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-workload-certs\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.569991 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.569902 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31b727dc-4050-4de6-ba38-88487799bda2-istiod-ca-cert\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.671010 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.670974 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-istio-envoy\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.671010 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.671013 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31b727dc-4050-4de6-ba38-88487799bda2-istio-token\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.671242 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.671029 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-workload-socket\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.671242 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.671172 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31b727dc-4050-4de6-ba38-88487799bda2-istio-podinfo\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.671242 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.671226 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-workload-certs\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.671421 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.671251 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31b727dc-4050-4de6-ba38-88487799bda2-istiod-ca-cert\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.671421 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.671349 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-istio-data\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.671421 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.671381 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w87w\" (UniqueName: \"kubernetes.io/projected/31b727dc-4050-4de6-ba38-88487799bda2-kube-api-access-2w87w\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.671421 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.671413 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-credential-socket\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.671623 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.671416 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-workload-socket\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.671623 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.671547 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-workload-certs\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.671751 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.671639 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-istio-data\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.671871 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.671851 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-credential-socket\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.672317 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.672293 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31b727dc-4050-4de6-ba38-88487799bda2-istiod-ca-cert\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.673573 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.673547 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-istio-envoy\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.673767 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.673750 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31b727dc-4050-4de6-ba38-88487799bda2-istio-podinfo\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.679043 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.679015 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31b727dc-4050-4de6-ba38-88487799bda2-istio-token\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.679457 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.679440 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w87w\" (UniqueName: \"kubernetes.io/projected/31b727dc-4050-4de6-ba38-88487799bda2-kube-api-access-2w87w\") pod \"e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.758385 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.758307 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:20:59.901385 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.901281 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42"] Jul 20 17:20:59.904278 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:20:59.904250 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31b727dc_4050_4de6_ba38_88487799bda2.slice/crio-986975f1206577b9c3ddbbb0e3012f00c974f1d9e4c7e8cf6406e14dc2a74690 WatchSource:0}: Error finding container 986975f1206577b9c3ddbbb0e3012f00c974f1d9e4c7e8cf6406e14dc2a74690: Status 404 returned error can't find the container with id 986975f1206577b9c3ddbbb0e3012f00c974f1d9e4c7e8cf6406e14dc2a74690 Jul 20 17:20:59.911918 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.911861 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:20:59.912042 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.912016 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:20:59.912130 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:20:59.912109 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:21:00.426626 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:00.426584 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" event={"ID":"31b727dc-4050-4de6-ba38-88487799bda2","Type":"ContainerStarted","Data":"181b94b570a9bab89a7d18cff8f81485e30191b77d2be7b992d0c564e9889c3e"} Jul 20 17:21:00.426837 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:00.426686 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" event={"ID":"31b727dc-4050-4de6-ba38-88487799bda2","Type":"ContainerStarted","Data":"986975f1206577b9c3ddbbb0e3012f00c974f1d9e4c7e8cf6406e14dc2a74690"} Jul 20 17:21:00.448762 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:00.448700 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" podStartSLOduration=1.4486742320000001 podStartE2EDuration="1.448674232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:20:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:21:00.447827645 +0000 UTC m=+2125.233826725" watchObservedRunningTime="2026-07-20 17:21:00.448674232 +0000 UTC m=+2125.234673311" Jul 20 17:21:00.759453 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:00.759363 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:21:00.764276 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:00.764252 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:21:01.433209 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:01.433178 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:21:01.434112 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:01.434096 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:21:16.508451 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:16.507912 2566 patch_prober.go:28] interesting pod/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": read tcp 10.134.0.2:37360->10.134.0.58:15021: read: connection reset by peer" start-of-body= Jul 20 17:21:16.508451 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:16.507996 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" podUID="31b727dc-4050-4de6-ba38-88487799bda2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": read tcp 10.134.0.2:37360->10.134.0.58:15021: read: connection reset by peer" Jul 20 17:21:17.529185 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:17.529154 2566 generic.go:356] "Generic (PLEG): container finished" podID="31b727dc-4050-4de6-ba38-88487799bda2" containerID="181b94b570a9bab89a7d18cff8f81485e30191b77d2be7b992d0c564e9889c3e" exitCode=0 Jul 20 17:21:17.529614 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:17.529232 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" event={"ID":"31b727dc-4050-4de6-ba38-88487799bda2","Type":"ContainerDied","Data":"181b94b570a9bab89a7d18cff8f81485e30191b77d2be7b992d0c564e9889c3e"} Jul 20 17:21:17.529682 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:17.529669 2566 scope.go:117] "RemoveContainer" containerID="181b94b570a9bab89a7d18cff8f81485e30191b77d2be7b992d0c564e9889c3e" Jul 20 17:21:17.530720 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:17.530689 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:21:17.530851 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:17.530788 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:21:17.530851 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:17.530838 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:21:18.538610 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:18.538580 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" event={"ID":"31b727dc-4050-4de6-ba38-88487799bda2","Type":"ContainerStarted","Data":"c0a4d81c7e14ac39927fa8a6eb65260a0500ac7e89c9d8d77527c38511dfb239"} Jul 20 17:21:18.758988 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:18.758947 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:21:18.759198 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:18.759173 2566 patch_prober.go:28] interesting pod/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 20 17:21:18.759282 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:18.759228 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" podUID="31b727dc-4050-4de6-ba38-88487799bda2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 20 17:21:19.758760 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:19.758712 2566 patch_prober.go:28] interesting pod/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 20 17:21:19.759155 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:19.758818 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" podUID="31b727dc-4050-4de6-ba38-88487799bda2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 20 17:21:20.759211 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:20.759175 2566 patch_prober.go:28] interesting pod/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 20 17:21:20.759677 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:20.759237 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" podUID="31b727dc-4050-4de6-ba38-88487799bda2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 20 17:21:21.758995 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:21.758957 2566 patch_prober.go:28] interesting pod/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 20 17:21:21.759175 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:21.759028 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" podUID="31b727dc-4050-4de6-ba38-88487799bda2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 20 17:21:22.762022 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:22.761988 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:21:22.762500 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:22.762389 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:21:22.763039 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:22.763021 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:21:36.001882 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:36.001852 2566 scope.go:117] "RemoveContainer" containerID="1aa10104373b1d3b5475db729d19dbf8701009bde1ae9011516e11b459669706" Jul 20 17:21:36.010818 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:21:36.010798 2566 scope.go:117] "RemoveContainer" containerID="119ee4cc705d053d194694a38e3ae3ccfbaf6c8842c951a1b4f88c596041f392" Jul 20 17:22:48.421080 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:48.421042 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1d7b9a-jj9d9"] Jul 20 17:22:48.424714 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:48.424696 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1d7b9a-jj9d9" Jul 20 17:22:48.426651 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:48.426633 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5jc2q\"" Jul 20 17:22:48.431926 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:48.431900 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1d7b9a-jj9d9"] Jul 20 17:22:48.490552 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:48.490515 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/826c9977-5014-40ae-a026-d8e59ab0bf3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1d7b9a-jj9d9\" (UID: \"826c9977-5014-40ae-a026-d8e59ab0bf3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1d7b9a-jj9d9" Jul 20 17:22:48.591384 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:48.591347 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/826c9977-5014-40ae-a026-d8e59ab0bf3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1d7b9a-jj9d9\" (UID: \"826c9977-5014-40ae-a026-d8e59ab0bf3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1d7b9a-jj9d9" Jul 20 17:22:48.591997 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:48.591975 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/826c9977-5014-40ae-a026-d8e59ab0bf3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1d7b9a-jj9d9\" (UID: \"826c9977-5014-40ae-a026-d8e59ab0bf3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1d7b9a-jj9d9" Jul 20 17:22:48.745611 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:48.745526 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1d7b9a-jj9d9" Jul 20 17:22:48.879401 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:48.879363 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1d7b9a-jj9d9"] Jul 20 17:22:48.881102 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:22:48.881073 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod826c9977_5014_40ae_a026_d8e59ab0bf3e.slice/crio-41a63c21e3f419bc6ba988b362753985cea7e632fd8bf64083bf9a43254f34df WatchSource:0}: Error finding container 41a63c21e3f419bc6ba988b362753985cea7e632fd8bf64083bf9a43254f34df: Status 404 returned error can't find the container with id 41a63c21e3f419bc6ba988b362753985cea7e632fd8bf64083bf9a43254f34df Jul 20 17:22:49.057097 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:49.056993 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1d7b9a-jj9d9" event={"ID":"826c9977-5014-40ae-a026-d8e59ab0bf3e","Type":"ContainerStarted","Data":"41a63c21e3f419bc6ba988b362753985cea7e632fd8bf64083bf9a43254f34df"} Jul 20 17:22:50.064479 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:50.064386 2566 generic.go:356] "Generic (PLEG): container finished" podID="826c9977-5014-40ae-a026-d8e59ab0bf3e" containerID="8a86fb643cc65424202bde979f3fac685df6b0dacaf7969d77a1dd977b290ff3" exitCode=0 Jul 20 17:22:50.064479 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:50.064436 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1d7b9a-jj9d9" event={"ID":"826c9977-5014-40ae-a026-d8e59ab0bf3e","Type":"ContainerDied","Data":"8a86fb643cc65424202bde979f3fac685df6b0dacaf7969d77a1dd977b290ff3"} Jul 20 17:22:51.199187 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:51.199163 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1d7b9a-jj9d9" Jul 20 17:22:51.316014 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:51.315977 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/826c9977-5014-40ae-a026-d8e59ab0bf3e-maas-api-service-ca\") pod \"826c9977-5014-40ae-a026-d8e59ab0bf3e\" (UID: \"826c9977-5014-40ae-a026-d8e59ab0bf3e\") " Jul 20 17:22:51.316326 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:51.316297 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/826c9977-5014-40ae-a026-d8e59ab0bf3e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "826c9977-5014-40ae-a026-d8e59ab0bf3e" (UID: "826c9977-5014-40ae-a026-d8e59ab0bf3e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:22:51.417065 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:51.417029 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/826c9977-5014-40ae-a026-d8e59ab0bf3e-maas-api-service-ca\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:22:52.077364 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:52.077274 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1d7b9a-jj9d9" Jul 20 17:22:52.077364 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:52.077299 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1d7b9a-jj9d9" event={"ID":"826c9977-5014-40ae-a026-d8e59ab0bf3e","Type":"ContainerDied","Data":"41a63c21e3f419bc6ba988b362753985cea7e632fd8bf64083bf9a43254f34df"} Jul 20 17:22:52.077364 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:22:52.077343 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41a63c21e3f419bc6ba988b362753985cea7e632fd8bf64083bf9a43254f34df" Jul 20 17:23:03.523483 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:03.523386 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1d7b9a-jj9d9"] Jul 20 17:23:03.526327 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:03.526298 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1d7b9a-jj9d9"] Jul 20 17:23:03.814458 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:03.814359 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="826c9977-5014-40ae-a026-d8e59ab0bf3e" path="/var/lib/kubelet/pods/826c9977-5014-40ae-a026-d8e59ab0bf3e/volumes" Jul 20 17:23:10.855013 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:10.854972 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0062e2-2pz7v"] Jul 20 17:23:10.855438 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:10.855423 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="826c9977-5014-40ae-a026-d8e59ab0bf3e" containerName="revoke-keys" Jul 20 17:23:10.855491 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:10.855441 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="826c9977-5014-40ae-a026-d8e59ab0bf3e" containerName="revoke-keys" Jul 20 17:23:10.855534 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:10.855509 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="826c9977-5014-40ae-a026-d8e59ab0bf3e" containerName="revoke-keys" Jul 20 17:23:10.858538 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:10.858520 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0062e2-2pz7v" Jul 20 17:23:10.860447 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:10.860432 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5jc2q\"" Jul 20 17:23:10.866517 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:10.866066 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0062e2-2pz7v"] Jul 20 17:23:10.900205 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:10.900172 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5446b3ac-e7a4-4140-901b-2c70e0c652ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0062e2-2pz7v\" (UID: \"5446b3ac-e7a4-4140-901b-2c70e0c652ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0062e2-2pz7v" Jul 20 17:23:11.000642 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:11.000605 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5446b3ac-e7a4-4140-901b-2c70e0c652ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0062e2-2pz7v\" (UID: \"5446b3ac-e7a4-4140-901b-2c70e0c652ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0062e2-2pz7v" Jul 20 17:23:11.001278 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:11.001259 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5446b3ac-e7a4-4140-901b-2c70e0c652ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0062e2-2pz7v\" (UID: \"5446b3ac-e7a4-4140-901b-2c70e0c652ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0062e2-2pz7v" Jul 20 17:23:11.170299 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:11.170215 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0062e2-2pz7v" Jul 20 17:23:11.314748 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:11.314712 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0062e2-2pz7v"] Jul 20 17:23:11.315249 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:23:11.315215 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5446b3ac_e7a4_4140_901b_2c70e0c652ef.slice/crio-018aa7c779b190b1a5a2bd1e3e7130afae75ab608940cfee7d3ed1d1143c0c72 WatchSource:0}: Error finding container 018aa7c779b190b1a5a2bd1e3e7130afae75ab608940cfee7d3ed1d1143c0c72: Status 404 returned error can't find the container with id 018aa7c779b190b1a5a2bd1e3e7130afae75ab608940cfee7d3ed1d1143c0c72 Jul 20 17:23:12.191563 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:12.191518 2566 generic.go:356] "Generic (PLEG): container finished" podID="5446b3ac-e7a4-4140-901b-2c70e0c652ef" containerID="ea10f37d52566c5cd5bdc584aed7ee836fb79adc0c944a458bb7019ae41eaf7c" exitCode=0 Jul 20 17:23:12.191935 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:12.191609 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0062e2-2pz7v" event={"ID":"5446b3ac-e7a4-4140-901b-2c70e0c652ef","Type":"ContainerDied","Data":"ea10f37d52566c5cd5bdc584aed7ee836fb79adc0c944a458bb7019ae41eaf7c"} Jul 20 17:23:12.191935 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:12.191663 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0062e2-2pz7v" event={"ID":"5446b3ac-e7a4-4140-901b-2c70e0c652ef","Type":"ContainerStarted","Data":"018aa7c779b190b1a5a2bd1e3e7130afae75ab608940cfee7d3ed1d1143c0c72"} Jul 20 17:23:13.328806 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:13.328761 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0062e2-2pz7v" Jul 20 17:23:13.421565 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:13.421529 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5446b3ac-e7a4-4140-901b-2c70e0c652ef-maas-api-service-ca\") pod \"5446b3ac-e7a4-4140-901b-2c70e0c652ef\" (UID: \"5446b3ac-e7a4-4140-901b-2c70e0c652ef\") " Jul 20 17:23:13.421909 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:13.421888 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5446b3ac-e7a4-4140-901b-2c70e0c652ef-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5446b3ac-e7a4-4140-901b-2c70e0c652ef" (UID: "5446b3ac-e7a4-4140-901b-2c70e0c652ef"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:23:13.523069 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:13.522978 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5446b3ac-e7a4-4140-901b-2c70e0c652ef-maas-api-service-ca\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:23:14.204314 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:14.204281 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0062e2-2pz7v" Jul 20 17:23:14.204489 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:14.204306 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0062e2-2pz7v" event={"ID":"5446b3ac-e7a4-4140-901b-2c70e0c652ef","Type":"ContainerDied","Data":"018aa7c779b190b1a5a2bd1e3e7130afae75ab608940cfee7d3ed1d1143c0c72"} Jul 20 17:23:14.204489 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:14.204362 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="018aa7c779b190b1a5a2bd1e3e7130afae75ab608940cfee7d3ed1d1143c0c72" Jul 20 17:23:25.942382 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:25.942332 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0062e2-2pz7v"] Jul 20 17:23:25.945256 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:25.945233 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0062e2-2pz7v"] Jul 20 17:23:27.814831 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:27.814765 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5446b3ac-e7a4-4140-901b-2c70e0c652ef" path="/var/lib/kubelet/pods/5446b3ac-e7a4-4140-901b-2c70e0c652ef/volumes" Jul 20 17:23:32.979145 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:32.979108 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42"] Jul 20 17:23:32.979521 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:32.979360 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" podUID="31b727dc-4050-4de6-ba38-88487799bda2" containerName="istio-proxy" containerID="cri-o://c0a4d81c7e14ac39927fa8a6eb65260a0500ac7e89c9d8d77527c38511dfb239" gracePeriod=30 Jul 20 17:23:36.043719 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:36.043685 2566 scope.go:117] "RemoveContainer" containerID="0e85d409cac138be3869f8241b4a91967dbb637e8aa56cefd00549217e345bdf" Jul 20 17:23:37.765048 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:37.765009 2566 patch_prober.go:28] interesting pod/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 20 17:23:37.765498 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:37.765087 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" podUID="31b727dc-4050-4de6-ba38-88487799bda2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 17:23:38.239460 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.239431 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:23:38.346043 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.346005 2566 generic.go:356] "Generic (PLEG): container finished" podID="31b727dc-4050-4de6-ba38-88487799bda2" containerID="c0a4d81c7e14ac39927fa8a6eb65260a0500ac7e89c9d8d77527c38511dfb239" exitCode=0 Jul 20 17:23:38.346230 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.346057 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" Jul 20 17:23:38.346230 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.346088 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" event={"ID":"31b727dc-4050-4de6-ba38-88487799bda2","Type":"ContainerDied","Data":"c0a4d81c7e14ac39927fa8a6eb65260a0500ac7e89c9d8d77527c38511dfb239"} Jul 20 17:23:38.346230 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.346145 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42" event={"ID":"31b727dc-4050-4de6-ba38-88487799bda2","Type":"ContainerDied","Data":"986975f1206577b9c3ddbbb0e3012f00c974f1d9e4c7e8cf6406e14dc2a74690"} Jul 20 17:23:38.346230 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.346167 2566 scope.go:117] "RemoveContainer" containerID="c0a4d81c7e14ac39927fa8a6eb65260a0500ac7e89c9d8d77527c38511dfb239" Jul 20 17:23:38.355217 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.355199 2566 scope.go:117] "RemoveContainer" containerID="181b94b570a9bab89a7d18cff8f81485e30191b77d2be7b992d0c564e9889c3e" Jul 20 17:23:38.360377 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.360357 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31b727dc-4050-4de6-ba38-88487799bda2-istio-token\") pod \"31b727dc-4050-4de6-ba38-88487799bda2\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " Jul 20 17:23:38.360466 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.360388 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-workload-certs\") pod \"31b727dc-4050-4de6-ba38-88487799bda2\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " Jul 20 17:23:38.360466 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.360452 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31b727dc-4050-4de6-ba38-88487799bda2-istiod-ca-cert\") pod \"31b727dc-4050-4de6-ba38-88487799bda2\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " Jul 20 17:23:38.360547 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.360483 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-istio-data\") pod \"31b727dc-4050-4de6-ba38-88487799bda2\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " Jul 20 17:23:38.360547 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.360539 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31b727dc-4050-4de6-ba38-88487799bda2-istio-podinfo\") pod \"31b727dc-4050-4de6-ba38-88487799bda2\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " Jul 20 17:23:38.360640 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.360567 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-istio-envoy\") pod \"31b727dc-4050-4de6-ba38-88487799bda2\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " Jul 20 17:23:38.360640 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.360608 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2w87w\" (UniqueName: \"kubernetes.io/projected/31b727dc-4050-4de6-ba38-88487799bda2-kube-api-access-2w87w\") pod \"31b727dc-4050-4de6-ba38-88487799bda2\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " Jul 20 17:23:38.360747 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.360687 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-credential-socket\") pod \"31b727dc-4050-4de6-ba38-88487799bda2\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " Jul 20 17:23:38.360747 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.360726 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-workload-socket\") pod \"31b727dc-4050-4de6-ba38-88487799bda2\" (UID: \"31b727dc-4050-4de6-ba38-88487799bda2\") " Jul 20 17:23:38.360881 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.360756 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "31b727dc-4050-4de6-ba38-88487799bda2" (UID: "31b727dc-4050-4de6-ba38-88487799bda2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:23:38.361110 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.360951 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31b727dc-4050-4de6-ba38-88487799bda2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "31b727dc-4050-4de6-ba38-88487799bda2" (UID: "31b727dc-4050-4de6-ba38-88487799bda2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:23:38.361110 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.361020 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "31b727dc-4050-4de6-ba38-88487799bda2" (UID: "31b727dc-4050-4de6-ba38-88487799bda2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:23:38.361260 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.361161 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "31b727dc-4050-4de6-ba38-88487799bda2" (UID: "31b727dc-4050-4de6-ba38-88487799bda2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:23:38.361260 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.361192 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31b727dc-4050-4de6-ba38-88487799bda2-istiod-ca-cert\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:23:38.361260 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.361212 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-credential-socket\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:23:38.361260 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.361226 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-workload-certs\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:23:38.361789 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.361747 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-istio-data" (OuterVolumeSpecName: "istio-data") pod "31b727dc-4050-4de6-ba38-88487799bda2" (UID: "31b727dc-4050-4de6-ba38-88487799bda2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:23:38.363103 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.363075 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "31b727dc-4050-4de6-ba38-88487799bda2" (UID: "31b727dc-4050-4de6-ba38-88487799bda2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:23:38.363186 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.363122 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31b727dc-4050-4de6-ba38-88487799bda2-istio-token" (OuterVolumeSpecName: "istio-token") pod "31b727dc-4050-4de6-ba38-88487799bda2" (UID: "31b727dc-4050-4de6-ba38-88487799bda2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:23:38.363267 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.363248 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/31b727dc-4050-4de6-ba38-88487799bda2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "31b727dc-4050-4de6-ba38-88487799bda2" (UID: "31b727dc-4050-4de6-ba38-88487799bda2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 17:23:38.363491 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.363467 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31b727dc-4050-4de6-ba38-88487799bda2-kube-api-access-2w87w" (OuterVolumeSpecName: "kube-api-access-2w87w") pod "31b727dc-4050-4de6-ba38-88487799bda2" (UID: "31b727dc-4050-4de6-ba38-88487799bda2"). InnerVolumeSpecName "kube-api-access-2w87w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:23:38.364894 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.364874 2566 scope.go:117] "RemoveContainer" containerID="c0a4d81c7e14ac39927fa8a6eb65260a0500ac7e89c9d8d77527c38511dfb239" Jul 20 17:23:38.365184 ip-10-0-130-18 kubenswrapper[2566]: E0720 17:23:38.365161 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0a4d81c7e14ac39927fa8a6eb65260a0500ac7e89c9d8d77527c38511dfb239\": container with ID starting with c0a4d81c7e14ac39927fa8a6eb65260a0500ac7e89c9d8d77527c38511dfb239 not found: ID does not exist" containerID="c0a4d81c7e14ac39927fa8a6eb65260a0500ac7e89c9d8d77527c38511dfb239" Jul 20 17:23:38.365234 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.365195 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0a4d81c7e14ac39927fa8a6eb65260a0500ac7e89c9d8d77527c38511dfb239"} err="failed to get container status \"c0a4d81c7e14ac39927fa8a6eb65260a0500ac7e89c9d8d77527c38511dfb239\": rpc error: code = NotFound desc = could not find container \"c0a4d81c7e14ac39927fa8a6eb65260a0500ac7e89c9d8d77527c38511dfb239\": container with ID starting with c0a4d81c7e14ac39927fa8a6eb65260a0500ac7e89c9d8d77527c38511dfb239 not found: ID does not exist" Jul 20 17:23:38.365234 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.365220 2566 scope.go:117] "RemoveContainer" containerID="181b94b570a9bab89a7d18cff8f81485e30191b77d2be7b992d0c564e9889c3e" Jul 20 17:23:38.365460 ip-10-0-130-18 kubenswrapper[2566]: E0720 17:23:38.365443 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"181b94b570a9bab89a7d18cff8f81485e30191b77d2be7b992d0c564e9889c3e\": container with ID starting with 181b94b570a9bab89a7d18cff8f81485e30191b77d2be7b992d0c564e9889c3e not found: ID does not exist" containerID="181b94b570a9bab89a7d18cff8f81485e30191b77d2be7b992d0c564e9889c3e" Jul 20 17:23:38.365514 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.365469 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"181b94b570a9bab89a7d18cff8f81485e30191b77d2be7b992d0c564e9889c3e"} err="failed to get container status \"181b94b570a9bab89a7d18cff8f81485e30191b77d2be7b992d0c564e9889c3e\": rpc error: code = NotFound desc = could not find container \"181b94b570a9bab89a7d18cff8f81485e30191b77d2be7b992d0c564e9889c3e\": container with ID starting with 181b94b570a9bab89a7d18cff8f81485e30191b77d2be7b992d0c564e9889c3e not found: ID does not exist" Jul 20 17:23:38.462721 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.462684 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31b727dc-4050-4de6-ba38-88487799bda2-istio-token\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:23:38.462721 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.462719 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-istio-data\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:23:38.462721 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.462728 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31b727dc-4050-4de6-ba38-88487799bda2-istio-podinfo\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:23:38.462987 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.462736 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-istio-envoy\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:23:38.462987 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.462745 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2w87w\" (UniqueName: \"kubernetes.io/projected/31b727dc-4050-4de6-ba38-88487799bda2-kube-api-access-2w87w\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:23:38.462987 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.462756 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31b727dc-4050-4de6-ba38-88487799bda2-workload-socket\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:23:38.669822 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.669791 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42"] Jul 20 17:23:38.675714 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:38.675687 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-0062e2-openshift-default-f4df95685-dbk42"] Jul 20 17:23:39.815278 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:39.815241 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31b727dc-4050-4de6-ba38-88487799bda2" path="/var/lib/kubelet/pods/31b727dc-4050-4de6-ba38-88487799bda2/volumes" Jul 20 17:23:57.493354 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:57.493308 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l"] Jul 20 17:23:57.493997 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:57.493897 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31b727dc-4050-4de6-ba38-88487799bda2" containerName="istio-proxy" Jul 20 17:23:57.493997 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:57.493919 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="31b727dc-4050-4de6-ba38-88487799bda2" containerName="istio-proxy" Jul 20 17:23:57.493997 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:57.493934 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31b727dc-4050-4de6-ba38-88487799bda2" containerName="istio-proxy" Jul 20 17:23:57.493997 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:57.493943 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="31b727dc-4050-4de6-ba38-88487799bda2" containerName="istio-proxy" Jul 20 17:23:57.493997 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:57.493972 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5446b3ac-e7a4-4140-901b-2c70e0c652ef" containerName="revoke-keys" Jul 20 17:23:57.493997 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:57.493980 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5446b3ac-e7a4-4140-901b-2c70e0c652ef" containerName="revoke-keys" Jul 20 17:23:57.494314 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:57.494088 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="31b727dc-4050-4de6-ba38-88487799bda2" containerName="istio-proxy" Jul 20 17:23:57.494314 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:57.494103 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5446b3ac-e7a4-4140-901b-2c70e0c652ef" containerName="revoke-keys" Jul 20 17:23:57.497231 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:57.497210 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l" Jul 20 17:23:57.499739 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:57.499718 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5jc2q\"" Jul 20 17:23:57.506527 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:57.506506 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l"] Jul 20 17:23:57.541334 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:57.541302 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d5be391-6d63-4e7d-95da-e9154b5c5b07-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l\" (UID: \"5d5be391-6d63-4e7d-95da-e9154b5c5b07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l" Jul 20 17:23:57.642055 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:57.642024 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d5be391-6d63-4e7d-95da-e9154b5c5b07-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l\" (UID: \"5d5be391-6d63-4e7d-95da-e9154b5c5b07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l" Jul 20 17:23:57.642605 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:57.642583 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d5be391-6d63-4e7d-95da-e9154b5c5b07-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l\" (UID: \"5d5be391-6d63-4e7d-95da-e9154b5c5b07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l" Jul 20 17:23:57.808612 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:57.808511 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l" Jul 20 17:23:57.949089 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:57.948889 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l"] Jul 20 17:23:57.951353 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:23:57.951325 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d5be391_6d63_4e7d_95da_e9154b5c5b07.slice/crio-63aae55d494af1a5b05801ae3d07065fe9c0c826e4c47645de7e6bf9f6c2e4db WatchSource:0}: Error finding container 63aae55d494af1a5b05801ae3d07065fe9c0c826e4c47645de7e6bf9f6c2e4db: Status 404 returned error can't find the container with id 63aae55d494af1a5b05801ae3d07065fe9c0c826e4c47645de7e6bf9f6c2e4db Jul 20 17:23:58.467582 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:58.467496 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l" event={"ID":"5d5be391-6d63-4e7d-95da-e9154b5c5b07","Type":"ContainerStarted","Data":"6b1b5b50b78d459f68d3622cedd2924baddd7b207865991212b5610c286d743b"} Jul 20 17:23:58.467582 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:58.467547 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l" event={"ID":"5d5be391-6d63-4e7d-95da-e9154b5c5b07","Type":"ContainerStarted","Data":"63aae55d494af1a5b05801ae3d07065fe9c0c826e4c47645de7e6bf9f6c2e4db"} Jul 20 17:23:58.482033 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:58.481948 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l" podStartSLOduration=1.4819354470000001 podStartE2EDuration="1.481935447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:23:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:23:58.481610038 +0000 UTC m=+2303.267609123" watchObservedRunningTime="2026-07-20 17:23:58.481935447 +0000 UTC m=+2303.267934528" Jul 20 17:23:59.474390 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:59.474351 2566 generic.go:356] "Generic (PLEG): container finished" podID="5d5be391-6d63-4e7d-95da-e9154b5c5b07" containerID="6b1b5b50b78d459f68d3622cedd2924baddd7b207865991212b5610c286d743b" exitCode=0 Jul 20 17:23:59.474825 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:23:59.474405 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l" event={"ID":"5d5be391-6d63-4e7d-95da-e9154b5c5b07","Type":"ContainerDied","Data":"6b1b5b50b78d459f68d3622cedd2924baddd7b207865991212b5610c286d743b"} Jul 20 17:24:00.611685 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:00.611658 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l" Jul 20 17:24:00.671052 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:00.671022 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d5be391-6d63-4e7d-95da-e9154b5c5b07-maas-api-service-ca\") pod \"5d5be391-6d63-4e7d-95da-e9154b5c5b07\" (UID: \"5d5be391-6d63-4e7d-95da-e9154b5c5b07\") " Jul 20 17:24:00.671371 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:00.671350 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d5be391-6d63-4e7d-95da-e9154b5c5b07-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5d5be391-6d63-4e7d-95da-e9154b5c5b07" (UID: "5d5be391-6d63-4e7d-95da-e9154b5c5b07"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:24:00.772632 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:00.772545 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d5be391-6d63-4e7d-95da-e9154b5c5b07-maas-api-service-ca\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:24:01.486343 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:01.486312 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l" Jul 20 17:24:01.486517 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:01.486349 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l" event={"ID":"5d5be391-6d63-4e7d-95da-e9154b5c5b07","Type":"ContainerDied","Data":"63aae55d494af1a5b05801ae3d07065fe9c0c826e4c47645de7e6bf9f6c2e4db"} Jul 20 17:24:01.486517 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:01.486396 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63aae55d494af1a5b05801ae3d07065fe9c0c826e4c47645de7e6bf9f6c2e4db" Jul 20 17:24:17.583441 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:17.583401 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l"] Jul 20 17:24:17.589148 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:17.589119 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77a9c7-zlm9l"] Jul 20 17:24:17.815517 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:17.815479 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d5be391-6d63-4e7d-95da-e9154b5c5b07" path="/var/lib/kubelet/pods/5d5be391-6d63-4e7d-95da-e9154b5c5b07/volumes" Jul 20 17:24:25.479202 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:25.479165 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c63878-fq2vg"] Jul 20 17:24:25.479650 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:25.479545 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d5be391-6d63-4e7d-95da-e9154b5c5b07" containerName="revoke-keys" Jul 20 17:24:25.479650 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:25.479556 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d5be391-6d63-4e7d-95da-e9154b5c5b07" containerName="revoke-keys" Jul 20 17:24:25.479650 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:25.479625 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d5be391-6d63-4e7d-95da-e9154b5c5b07" containerName="revoke-keys" Jul 20 17:24:25.479650 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:25.479636 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="31b727dc-4050-4de6-ba38-88487799bda2" containerName="istio-proxy" Jul 20 17:24:25.482953 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:25.482934 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c63878-fq2vg" Jul 20 17:24:25.485663 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:25.485639 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5jc2q\"" Jul 20 17:24:25.502828 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:25.502798 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c63878-fq2vg"] Jul 20 17:24:25.603903 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:25.603871 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c20c8d8-117b-45b3-80b9-8346ecdb1ea5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c63878-fq2vg\" (UID: \"3c20c8d8-117b-45b3-80b9-8346ecdb1ea5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c63878-fq2vg" Jul 20 17:24:25.704934 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:25.704895 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c20c8d8-117b-45b3-80b9-8346ecdb1ea5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c63878-fq2vg\" (UID: \"3c20c8d8-117b-45b3-80b9-8346ecdb1ea5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c63878-fq2vg" Jul 20 17:24:25.705513 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:25.705493 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c20c8d8-117b-45b3-80b9-8346ecdb1ea5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c63878-fq2vg\" (UID: \"3c20c8d8-117b-45b3-80b9-8346ecdb1ea5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c63878-fq2vg" Jul 20 17:24:25.792818 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:25.792724 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c63878-fq2vg" Jul 20 17:24:25.936855 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:25.936826 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c63878-fq2vg"] Jul 20 17:24:25.938461 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:24:25.938434 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c20c8d8_117b_45b3_80b9_8346ecdb1ea5.slice/crio-297df3cf577802090f46241f0966b56bafe1ba2129e01f301a5644269bd3df8f WatchSource:0}: Error finding container 297df3cf577802090f46241f0966b56bafe1ba2129e01f301a5644269bd3df8f: Status 404 returned error can't find the container with id 297df3cf577802090f46241f0966b56bafe1ba2129e01f301a5644269bd3df8f Jul 20 17:24:26.632428 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:26.632390 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c63878-fq2vg" event={"ID":"3c20c8d8-117b-45b3-80b9-8346ecdb1ea5","Type":"ContainerStarted","Data":"52222e722e1718fd084fa1cef6277791712a66e5da303a388578760a85b6ba91"} Jul 20 17:24:26.632826 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:26.632458 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c63878-fq2vg" event={"ID":"3c20c8d8-117b-45b3-80b9-8346ecdb1ea5","Type":"ContainerStarted","Data":"297df3cf577802090f46241f0966b56bafe1ba2129e01f301a5644269bd3df8f"} Jul 20 17:24:26.655990 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:26.655933 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c63878-fq2vg" podStartSLOduration=1.655917281 podStartE2EDuration="1.655917281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:24:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:24:26.655824947 +0000 UTC m=+2331.441824033" watchObservedRunningTime="2026-07-20 17:24:26.655917281 +0000 UTC m=+2331.441916385" Jul 20 17:24:27.639340 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:27.639307 2566 generic.go:356] "Generic (PLEG): container finished" podID="3c20c8d8-117b-45b3-80b9-8346ecdb1ea5" containerID="52222e722e1718fd084fa1cef6277791712a66e5da303a388578760a85b6ba91" exitCode=0 Jul 20 17:24:27.639725 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:27.639399 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c63878-fq2vg" event={"ID":"3c20c8d8-117b-45b3-80b9-8346ecdb1ea5","Type":"ContainerDied","Data":"52222e722e1718fd084fa1cef6277791712a66e5da303a388578760a85b6ba91"} Jul 20 17:24:28.779197 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:28.779137 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c63878-fq2vg" Jul 20 17:24:28.935986 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:28.935895 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c20c8d8-117b-45b3-80b9-8346ecdb1ea5-maas-api-service-ca\") pod \"3c20c8d8-117b-45b3-80b9-8346ecdb1ea5\" (UID: \"3c20c8d8-117b-45b3-80b9-8346ecdb1ea5\") " Jul 20 17:24:28.936258 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:28.936236 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c20c8d8-117b-45b3-80b9-8346ecdb1ea5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3c20c8d8-117b-45b3-80b9-8346ecdb1ea5" (UID: "3c20c8d8-117b-45b3-80b9-8346ecdb1ea5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:24:29.037556 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:29.037520 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c20c8d8-117b-45b3-80b9-8346ecdb1ea5-maas-api-service-ca\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:24:29.652639 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:29.652609 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c63878-fq2vg" Jul 20 17:24:29.652869 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:29.652642 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c63878-fq2vg" event={"ID":"3c20c8d8-117b-45b3-80b9-8346ecdb1ea5","Type":"ContainerDied","Data":"297df3cf577802090f46241f0966b56bafe1ba2129e01f301a5644269bd3df8f"} Jul 20 17:24:29.652869 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:29.652689 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="297df3cf577802090f46241f0966b56bafe1ba2129e01f301a5644269bd3df8f" Jul 20 17:24:45.557402 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:45.557364 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c63878-fq2vg"] Jul 20 17:24:45.564296 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:45.564260 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c63878-fq2vg"] Jul 20 17:24:45.816018 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:24:45.815933 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c20c8d8-117b-45b3-80b9-8346ecdb1ea5" path="/var/lib/kubelet/pods/3c20c8d8-117b-45b3-80b9-8346ecdb1ea5/volumes" Jul 20 17:25:35.937804 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:35.937756 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 17:25:35.946019 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:35.945994 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 17:25:36.082031 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:36.081999 2566 scope.go:117] "RemoveContainer" containerID="f02a7678da086885914cfbabf7a36b993f6315e882dfe8281bb59322628fcebb" Jul 20 17:25:49.891889 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:49.891856 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj"] Jul 20 17:25:49.892508 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:49.892418 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c20c8d8-117b-45b3-80b9-8346ecdb1ea5" containerName="revoke-keys" Jul 20 17:25:49.892508 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:49.892439 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c20c8d8-117b-45b3-80b9-8346ecdb1ea5" containerName="revoke-keys" Jul 20 17:25:49.892637 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:49.892567 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c20c8d8-117b-45b3-80b9-8346ecdb1ea5" containerName="revoke-keys" Jul 20 17:25:49.895694 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:49.895671 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:49.898062 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:49.898030 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 20 17:25:49.898183 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:49.898127 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-a24b97-openshift-default-dockercfg-27cms\"" Jul 20 17:25:49.907719 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:49.907691 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj"] Jul 20 17:25:49.986048 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:49.986006 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-data\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:49.986048 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:49.986049 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khljw\" (UniqueName: \"kubernetes.io/projected/7f0226da-5df5-46c3-8f02-6d692dd1f906-kube-api-access-khljw\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:49.986351 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:49.986094 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-podinfo\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:49.986351 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:49.986233 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-workload-socket\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:49.986475 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:49.986454 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-credential-socket\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:49.986535 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:49.986491 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-token\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:49.986593 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:49.986537 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f0226da-5df5-46c3-8f02-6d692dd1f906-istiod-ca-cert\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:49.986593 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:49.986568 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-workload-certs\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:49.986706 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:49.986616 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-envoy\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.087169 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.087134 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-workload-socket\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.087390 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.087205 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-credential-socket\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.087390 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.087225 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-token\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.087390 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.087242 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f0226da-5df5-46c3-8f02-6d692dd1f906-istiod-ca-cert\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.087390 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.087270 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-workload-certs\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.087390 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.087293 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-envoy\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.087390 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.087325 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-data\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.087390 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.087363 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khljw\" (UniqueName: \"kubernetes.io/projected/7f0226da-5df5-46c3-8f02-6d692dd1f906-kube-api-access-khljw\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.087745 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.087425 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-podinfo\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.087745 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.087624 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-workload-socket\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.087898 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.087875 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-workload-certs\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.087944 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.087909 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-data\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.088079 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.088056 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f0226da-5df5-46c3-8f02-6d692dd1f906-istiod-ca-cert\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.088277 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.088261 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-credential-socket\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.090067 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.090041 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-envoy\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.090197 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.090178 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-podinfo\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.099268 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.099242 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-token\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.100115 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.100092 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khljw\" (UniqueName: \"kubernetes.io/projected/7f0226da-5df5-46c3-8f02-6d692dd1f906-kube-api-access-khljw\") pod \"e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.206560 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.206465 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:50.352727 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.352697 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj"] Jul 20 17:25:50.353954 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:25:50.353921 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f0226da_5df5_46c3_8f02_6d692dd1f906.slice/crio-74d007d72ad7ccfe3ea997c8109cc260a06c68cb12df1cef9c25c5899e9eaa5c WatchSource:0}: Error finding container 74d007d72ad7ccfe3ea997c8109cc260a06c68cb12df1cef9c25c5899e9eaa5c: Status 404 returned error can't find the container with id 74d007d72ad7ccfe3ea997c8109cc260a06c68cb12df1cef9c25c5899e9eaa5c Jul 20 17:25:50.356057 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.356025 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:25:50.356151 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.356090 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:25:50.356151 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:50.356117 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:25:51.117751 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:51.117711 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" event={"ID":"7f0226da-5df5-46c3-8f02-6d692dd1f906","Type":"ContainerStarted","Data":"07908ebf410f7444b8839f7d2077f1d6981337622d0810ba64dc30696c36eaa5"} Jul 20 17:25:51.118168 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:51.117822 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" event={"ID":"7f0226da-5df5-46c3-8f02-6d692dd1f906","Type":"ContainerStarted","Data":"74d007d72ad7ccfe3ea997c8109cc260a06c68cb12df1cef9c25c5899e9eaa5c"} Jul 20 17:25:51.138667 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:51.138615 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" podStartSLOduration=2.138600775 podStartE2EDuration="2.138600775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:25:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:25:51.137171027 +0000 UTC m=+2415.923170107" watchObservedRunningTime="2026-07-20 17:25:51.138600775 +0000 UTC m=+2415.924599853" Jul 20 17:25:51.207180 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:51.207143 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:51.211966 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:51.211942 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:52.123556 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:52.123527 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:25:52.124640 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:25:52.124623 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:26:08.166968 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:08.166922 2566 patch_prober.go:28] interesting pod/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 20 17:26:08.167480 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:08.167031 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" podUID="7f0226da-5df5-46c3-8f02-6d692dd1f906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 20 17:26:10.235493 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:10.235462 2566 generic.go:356] "Generic (PLEG): container finished" podID="7f0226da-5df5-46c3-8f02-6d692dd1f906" containerID="07908ebf410f7444b8839f7d2077f1d6981337622d0810ba64dc30696c36eaa5" exitCode=0 Jul 20 17:26:10.236002 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:10.235541 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" event={"ID":"7f0226da-5df5-46c3-8f02-6d692dd1f906","Type":"ContainerDied","Data":"07908ebf410f7444b8839f7d2077f1d6981337622d0810ba64dc30696c36eaa5"} Jul 20 17:26:10.236002 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:10.235981 2566 scope.go:117] "RemoveContainer" containerID="07908ebf410f7444b8839f7d2077f1d6981337622d0810ba64dc30696c36eaa5" Jul 20 17:26:10.237005 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:10.236976 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:26:10.237091 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:10.237040 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:26:10.237091 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:10.237073 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:26:11.243419 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:11.243385 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" event={"ID":"7f0226da-5df5-46c3-8f02-6d692dd1f906","Type":"ContainerStarted","Data":"6652a9be57b3a12dcebb016f16f9800cdaca9bedea113701cd717862be9e7dd3"} Jul 20 17:26:12.207069 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:12.207028 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:26:12.207308 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:12.207282 2566 patch_prober.go:28] interesting pod/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 20 17:26:12.207366 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:12.207343 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" podUID="7f0226da-5df5-46c3-8f02-6d692dd1f906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 20 17:26:13.207853 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:13.207818 2566 patch_prober.go:28] interesting pod/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 20 17:26:13.208447 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:13.208415 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" podUID="7f0226da-5df5-46c3-8f02-6d692dd1f906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 20 17:26:14.207604 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:14.207567 2566 patch_prober.go:28] interesting pod/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 20 17:26:14.207825 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:14.207640 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" podUID="7f0226da-5df5-46c3-8f02-6d692dd1f906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 20 17:26:15.207480 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:15.207446 2566 patch_prober.go:28] interesting pod/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 20 17:26:15.207915 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:15.207500 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" podUID="7f0226da-5df5-46c3-8f02-6d692dd1f906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 20 17:26:16.211207 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:16.211178 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:26:16.211671 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:16.211454 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:26:16.212242 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:26:16.212225 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:29:36.131384 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:29:36.131347 2566 scope.go:117] "RemoveContainer" containerID="ea10f37d52566c5cd5bdc584aed7ee836fb79adc0c944a458bb7019ae41eaf7c" Jul 20 17:29:36.140724 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:29:36.140702 2566 scope.go:117] "RemoveContainer" containerID="8a86fb643cc65424202bde979f3fac685df6b0dacaf7969d77a1dd977b290ff3" Jul 20 17:30:00.129636 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.129594 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a24b97-29742810-zdbgt"] Jul 20 17:30:00.133341 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.133319 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742810-xsccz"] Jul 20 17:30:00.133475 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.133434 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a24b97-29742810-zdbgt" Jul 20 17:30:00.135441 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.135418 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5jc2q\"" Jul 20 17:30:00.136412 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.136397 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f5995d-29742810-d8zk7"] Jul 20 17:30:00.136510 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.136495 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742810-xsccz" Jul 20 17:30:00.139548 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.139530 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f5995d-29742810-d8zk7" Jul 20 17:30:00.143633 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.143575 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742810-xsccz"] Jul 20 17:30:00.146174 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.146154 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a24b97-29742810-zdbgt" Jul 20 17:30:00.146422 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.146404 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a24b97-29742810-zdbgt"] Jul 20 17:30:00.149472 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.149451 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f5995d-29742810-d8zk7"] Jul 20 17:30:00.157305 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.157275 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742810-xsccz" Jul 20 17:30:00.194629 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.194576 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f5995d-29742810-d8zk7" Jul 20 17:30:00.365902 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.365872 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f5995d-29742810-d8zk7"] Jul 20 17:30:00.367318 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:30:00.367292 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod476890db_318f_46a2_8e06_cd78ffc0558f.slice/crio-eec9173305855bb57e86219810efb4f1baa1b59c746f3eb0518b9ec926637a03 WatchSource:0}: Error finding container eec9173305855bb57e86219810efb4f1baa1b59c746f3eb0518b9ec926637a03: Status 404 returned error can't find the container with id eec9173305855bb57e86219810efb4f1baa1b59c746f3eb0518b9ec926637a03 Jul 20 17:30:00.535899 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.535875 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742810-xsccz"] Jul 20 17:30:00.538096 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:30:00.538054 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3914a7ec_0305_4aba_83ba_e42c5f1714b5.slice/crio-3b12d6e8926cbdd9b948a0b63769d7b67b24549cb2d3984a7b5930af90f9d98a WatchSource:0}: Error finding container 3b12d6e8926cbdd9b948a0b63769d7b67b24549cb2d3984a7b5930af90f9d98a: Status 404 returned error can't find the container with id 3b12d6e8926cbdd9b948a0b63769d7b67b24549cb2d3984a7b5930af90f9d98a Jul 20 17:30:00.539114 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.539088 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a24b97-29742810-zdbgt"] Jul 20 17:30:00.539421 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:30:00.539224 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb71db8d2_4741_4662_8b90_07b59710dbf3.slice/crio-23b43ee7580bfb0b2bf140a38953ced51738ec4a4008de0c464beb8f2173b97d WatchSource:0}: Error finding container 23b43ee7580bfb0b2bf140a38953ced51738ec4a4008de0c464beb8f2173b97d: Status 404 returned error can't find the container with id 23b43ee7580bfb0b2bf140a38953ced51738ec4a4008de0c464beb8f2173b97d Jul 20 17:30:00.545936 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.545912 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f5995d-29742810-d8zk7" event={"ID":"476890db-318f-46a2-8e06-cd78ffc0558f","Type":"ContainerStarted","Data":"632fbea9a2407d3d05320b69f9e4fcf29dadd918c26670c85011df7a999aa0a2"} Jul 20 17:30:00.546023 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.545956 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f5995d-29742810-d8zk7" event={"ID":"476890db-318f-46a2-8e06-cd78ffc0558f","Type":"ContainerStarted","Data":"eec9173305855bb57e86219810efb4f1baa1b59c746f3eb0518b9ec926637a03"} Jul 20 17:30:00.563976 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:00.563928 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f5995d-29742810-d8zk7" podStartSLOduration=0.56390697 podStartE2EDuration="563.90697ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:30:00.561500085 +0000 UTC m=+2665.347499165" watchObservedRunningTime="2026-07-20 17:30:00.56390697 +0000 UTC m=+2665.349906049" Jul 20 17:30:01.554580 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:01.554545 2566 generic.go:356] "Generic (PLEG): container finished" podID="476890db-318f-46a2-8e06-cd78ffc0558f" containerID="632fbea9a2407d3d05320b69f9e4fcf29dadd918c26670c85011df7a999aa0a2" exitCode=0 Jul 20 17:30:01.554999 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:01.554621 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f5995d-29742810-d8zk7" event={"ID":"476890db-318f-46a2-8e06-cd78ffc0558f","Type":"ContainerDied","Data":"632fbea9a2407d3d05320b69f9e4fcf29dadd918c26670c85011df7a999aa0a2"} Jul 20 17:30:01.556151 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:01.556128 2566 generic.go:356] "Generic (PLEG): container finished" podID="3914a7ec-0305-4aba-83ba-e42c5f1714b5" containerID="9beaabc5e9181468880e894034cb234f3320879cf68e2bb0b571fbb76c0e3c27" exitCode=0 Jul 20 17:30:01.556277 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:01.556186 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a24b97-29742810-zdbgt" event={"ID":"3914a7ec-0305-4aba-83ba-e42c5f1714b5","Type":"ContainerDied","Data":"9beaabc5e9181468880e894034cb234f3320879cf68e2bb0b571fbb76c0e3c27"} Jul 20 17:30:01.556277 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:01.556238 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a24b97-29742810-zdbgt" event={"ID":"3914a7ec-0305-4aba-83ba-e42c5f1714b5","Type":"ContainerStarted","Data":"3b12d6e8926cbdd9b948a0b63769d7b67b24549cb2d3984a7b5930af90f9d98a"} Jul 20 17:30:01.557675 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:01.557654 2566 generic.go:356] "Generic (PLEG): container finished" podID="b71db8d2-4741-4662-8b90-07b59710dbf3" containerID="f2600c3c5934fe7074b7fc14d0668c59d6408d0033fa335a6886a62dbb91dd6e" exitCode=0 Jul 20 17:30:01.557768 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:01.557699 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742810-xsccz" event={"ID":"b71db8d2-4741-4662-8b90-07b59710dbf3","Type":"ContainerDied","Data":"f2600c3c5934fe7074b7fc14d0668c59d6408d0033fa335a6886a62dbb91dd6e"} Jul 20 17:30:01.557768 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:01.557746 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742810-xsccz" event={"ID":"b71db8d2-4741-4662-8b90-07b59710dbf3","Type":"ContainerStarted","Data":"23b43ee7580bfb0b2bf140a38953ced51738ec4a4008de0c464beb8f2173b97d"} Jul 20 17:30:02.738976 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:02.738940 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742810-xsccz" Jul 20 17:30:02.742188 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:02.742164 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f5995d-29742810-d8zk7" Jul 20 17:30:02.745788 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:02.745758 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a24b97-29742810-zdbgt" Jul 20 17:30:03.571754 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:03.571726 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a24b97-29742810-zdbgt" Jul 20 17:30:03.571961 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:03.571761 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a24b97-29742810-zdbgt" event={"ID":"3914a7ec-0305-4aba-83ba-e42c5f1714b5","Type":"ContainerDied","Data":"3b12d6e8926cbdd9b948a0b63769d7b67b24549cb2d3984a7b5930af90f9d98a"} Jul 20 17:30:03.571961 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:03.571831 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b12d6e8926cbdd9b948a0b63769d7b67b24549cb2d3984a7b5930af90f9d98a" Jul 20 17:30:03.573137 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:03.573116 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742810-xsccz" Jul 20 17:30:03.573274 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:03.573173 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742810-xsccz" event={"ID":"b71db8d2-4741-4662-8b90-07b59710dbf3","Type":"ContainerDied","Data":"23b43ee7580bfb0b2bf140a38953ced51738ec4a4008de0c464beb8f2173b97d"} Jul 20 17:30:03.573274 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:03.573222 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23b43ee7580bfb0b2bf140a38953ced51738ec4a4008de0c464beb8f2173b97d" Jul 20 17:30:03.574601 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:03.574577 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f5995d-29742810-d8zk7" event={"ID":"476890db-318f-46a2-8e06-cd78ffc0558f","Type":"ContainerDied","Data":"eec9173305855bb57e86219810efb4f1baa1b59c746f3eb0518b9ec926637a03"} Jul 20 17:30:03.574704 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:03.574614 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f5995d-29742810-d8zk7" Jul 20 17:30:03.574704 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:03.574617 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eec9173305855bb57e86219810efb4f1baa1b59c746f3eb0518b9ec926637a03" Jul 20 17:30:35.977651 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:35.977569 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 17:30:35.986887 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:35.986866 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5n5k_48d72ddd-0615-4d29-875a-664eda1f2f88/ovn-acl-logging/0.log" Jul 20 17:30:36.165872 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:36.165843 2566 scope.go:117] "RemoveContainer" containerID="6b1b5b50b78d459f68d3622cedd2924baddd7b207865991212b5610c286d743b" Jul 20 17:30:36.175207 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:30:36.175186 2566 scope.go:117] "RemoveContainer" containerID="52222e722e1718fd084fa1cef6277791712a66e5da303a388578760a85b6ba91" Jul 20 17:31:58.908367 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:58.908272 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7269-140156648352560"] Jul 20 17:31:58.908990 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:58.908874 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b71db8d2-4741-4662-8b90-07b59710dbf3" containerName="cleanup" Jul 20 17:31:58.908990 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:58.908896 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b71db8d2-4741-4662-8b90-07b59710dbf3" containerName="cleanup" Jul 20 17:31:58.908990 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:58.908928 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3914a7ec-0305-4aba-83ba-e42c5f1714b5" containerName="cleanup" Jul 20 17:31:58.908990 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:58.908937 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3914a7ec-0305-4aba-83ba-e42c5f1714b5" containerName="cleanup" Jul 20 17:31:58.908990 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:58.908957 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="476890db-318f-46a2-8e06-cd78ffc0558f" containerName="cleanup" Jul 20 17:31:58.908990 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:58.908966 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="476890db-318f-46a2-8e06-cd78ffc0558f" containerName="cleanup" Jul 20 17:31:58.909308 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:58.909052 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3914a7ec-0305-4aba-83ba-e42c5f1714b5" containerName="cleanup" Jul 20 17:31:58.909308 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:58.909065 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="476890db-318f-46a2-8e06-cd78ffc0558f" containerName="cleanup" Jul 20 17:31:58.909308 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:58.909074 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b71db8d2-4741-4662-8b90-07b59710dbf3" containerName="cleanup" Jul 20 17:31:58.912222 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:58.912199 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648352560" Jul 20 17:31:58.914416 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:58.914399 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-f7psf\"" Jul 20 17:31:58.919209 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:58.919185 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7269-140156648352560"] Jul 20 17:31:58.974886 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:58.974845 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nktqv\" (UniqueName: \"kubernetes.io/projected/3307b242-e148-4d0c-a552-c29aa73a47f1-kube-api-access-nktqv\") pod \"test-curl-7269-140156648352560\" (UID: \"3307b242-e148-4d0c-a552-c29aa73a47f1\") " pod="opendatahub/test-curl-7269-140156648352560" Jul 20 17:31:59.075594 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:59.075552 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nktqv\" (UniqueName: \"kubernetes.io/projected/3307b242-e148-4d0c-a552-c29aa73a47f1-kube-api-access-nktqv\") pod \"test-curl-7269-140156648352560\" (UID: \"3307b242-e148-4d0c-a552-c29aa73a47f1\") " pod="opendatahub/test-curl-7269-140156648352560" Jul 20 17:31:59.084062 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:59.084034 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nktqv\" (UniqueName: \"kubernetes.io/projected/3307b242-e148-4d0c-a552-c29aa73a47f1-kube-api-access-nktqv\") pod \"test-curl-7269-140156648352560\" (UID: \"3307b242-e148-4d0c-a552-c29aa73a47f1\") " pod="opendatahub/test-curl-7269-140156648352560" Jul 20 17:31:59.222537 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:59.222448 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648352560" Jul 20 17:31:59.360570 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:59.360542 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7269-140156648352560"] Jul 20 17:31:59.362741 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:31:59.362702 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3307b242_e148_4d0c_a552_c29aa73a47f1.slice/crio-b26a4f7830f677539cf7065b57c2e478772089e6c7321152b17edf9e2ef93c25 WatchSource:0}: Error finding container b26a4f7830f677539cf7065b57c2e478772089e6c7321152b17edf9e2ef93c25: Status 404 returned error can't find the container with id b26a4f7830f677539cf7065b57c2e478772089e6c7321152b17edf9e2ef93c25 Jul 20 17:31:59.364197 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:31:59.364175 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 17:32:00.258377 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:00.258344 2566 generic.go:356] "Generic (PLEG): container finished" podID="3307b242-e148-4d0c-a552-c29aa73a47f1" containerID="9290f6e5ff375d474b811c7fd63d1c5477055954f89bd6802f6dd9f384051cbe" exitCode=0 Jul 20 17:32:00.258820 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:00.258435 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7269-140156648352560" event={"ID":"3307b242-e148-4d0c-a552-c29aa73a47f1","Type":"ContainerDied","Data":"9290f6e5ff375d474b811c7fd63d1c5477055954f89bd6802f6dd9f384051cbe"} Jul 20 17:32:00.258820 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:00.258475 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7269-140156648352560" event={"ID":"3307b242-e148-4d0c-a552-c29aa73a47f1","Type":"ContainerStarted","Data":"b26a4f7830f677539cf7065b57c2e478772089e6c7321152b17edf9e2ef93c25"} Jul 20 17:32:01.404694 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:01.404665 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648352560" Jul 20 17:32:01.451100 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:01.451064 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7269-140156648352560_3307b242-e148-4d0c-a552-c29aa73a47f1/test-curl-7269-140156648352560/0.log" Jul 20 17:32:01.479562 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:01.479520 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7269-140156648352560"] Jul 20 17:32:01.482993 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:01.482960 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7269-140156648352560"] Jul 20 17:32:01.492490 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:01.492463 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nktqv\" (UniqueName: \"kubernetes.io/projected/3307b242-e148-4d0c-a552-c29aa73a47f1-kube-api-access-nktqv\") pod \"3307b242-e148-4d0c-a552-c29aa73a47f1\" (UID: \"3307b242-e148-4d0c-a552-c29aa73a47f1\") " Jul 20 17:32:01.494859 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:01.494834 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3307b242-e148-4d0c-a552-c29aa73a47f1-kube-api-access-nktqv" (OuterVolumeSpecName: "kube-api-access-nktqv") pod "3307b242-e148-4d0c-a552-c29aa73a47f1" (UID: "3307b242-e148-4d0c-a552-c29aa73a47f1"). InnerVolumeSpecName "kube-api-access-nktqv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:32:01.593599 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:01.593564 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nktqv\" (UniqueName: \"kubernetes.io/projected/3307b242-e148-4d0c-a552-c29aa73a47f1-kube-api-access-nktqv\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:32:01.610309 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:01.610272 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7269-140156648352944"] Jul 20 17:32:01.610711 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:01.610697 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3307b242-e148-4d0c-a552-c29aa73a47f1" containerName="test-curl-7269-140156648352560" Jul 20 17:32:01.610757 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:01.610713 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3307b242-e148-4d0c-a552-c29aa73a47f1" containerName="test-curl-7269-140156648352560" Jul 20 17:32:01.610823 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:01.610814 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3307b242-e148-4d0c-a552-c29aa73a47f1" containerName="test-curl-7269-140156648352560" Jul 20 17:32:01.614061 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:01.614044 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648352944" Jul 20 17:32:01.619002 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:01.618976 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7269-140156648352944"] Jul 20 17:32:01.694149 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:01.694103 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgzwp\" (UniqueName: \"kubernetes.io/projected/1364a297-1675-430d-9fea-9cb25925923a-kube-api-access-lgzwp\") pod \"test-curl-7269-140156648352944\" (UID: \"1364a297-1675-430d-9fea-9cb25925923a\") " pod="opendatahub/test-curl-7269-140156648352944" Jul 20 17:32:01.795156 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:01.795119 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgzwp\" (UniqueName: \"kubernetes.io/projected/1364a297-1675-430d-9fea-9cb25925923a-kube-api-access-lgzwp\") pod \"test-curl-7269-140156648352944\" (UID: \"1364a297-1675-430d-9fea-9cb25925923a\") " pod="opendatahub/test-curl-7269-140156648352944" Jul 20 17:32:01.804180 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:01.804152 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgzwp\" (UniqueName: \"kubernetes.io/projected/1364a297-1675-430d-9fea-9cb25925923a-kube-api-access-lgzwp\") pod \"test-curl-7269-140156648352944\" (UID: \"1364a297-1675-430d-9fea-9cb25925923a\") " pod="opendatahub/test-curl-7269-140156648352944" Jul 20 17:32:01.816288 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:01.816246 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3307b242-e148-4d0c-a552-c29aa73a47f1" path="/var/lib/kubelet/pods/3307b242-e148-4d0c-a552-c29aa73a47f1/volumes" Jul 20 17:32:01.925112 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:01.925008 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648352944" Jul 20 17:32:02.070127 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:02.070101 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7269-140156648352944"] Jul 20 17:32:02.071804 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:32:02.071758 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1364a297_1675_430d_9fea_9cb25925923a.slice/crio-2fdd57683efd062884f89720dafb6e64d513c975163dcf8253e96dda5bfce634 WatchSource:0}: Error finding container 2fdd57683efd062884f89720dafb6e64d513c975163dcf8253e96dda5bfce634: Status 404 returned error can't find the container with id 2fdd57683efd062884f89720dafb6e64d513c975163dcf8253e96dda5bfce634 Jul 20 17:32:02.272144 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:02.272047 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648352560" Jul 20 17:32:02.272144 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:02.272070 2566 scope.go:117] "RemoveContainer" containerID="9290f6e5ff375d474b811c7fd63d1c5477055954f89bd6802f6dd9f384051cbe" Jul 20 17:32:02.273597 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:02.273566 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7269-140156648352944" event={"ID":"1364a297-1675-430d-9fea-9cb25925923a","Type":"ContainerStarted","Data":"2fdd57683efd062884f89720dafb6e64d513c975163dcf8253e96dda5bfce634"} Jul 20 17:32:03.280872 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:03.280833 2566 generic.go:356] "Generic (PLEG): container finished" podID="1364a297-1675-430d-9fea-9cb25925923a" containerID="f2177b3d6d04caa89ccdb831546ed5988ccfa4f63cf7b84f4840f59f554440a8" exitCode=0 Jul 20 17:32:03.281305 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:03.280866 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7269-140156648352944" event={"ID":"1364a297-1675-430d-9fea-9cb25925923a","Type":"ContainerDied","Data":"f2177b3d6d04caa89ccdb831546ed5988ccfa4f63cf7b84f4840f59f554440a8"} Jul 20 17:32:04.426125 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:04.426102 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648352944" Jul 20 17:32:04.449516 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:04.449493 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7269-140156648352944_1364a297-1675-430d-9fea-9cb25925923a/test-curl-7269-140156648352944/0.log" Jul 20 17:32:04.482722 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:04.482682 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7269-140156648352944"] Jul 20 17:32:04.485867 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:04.485833 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7269-140156648352944"] Jul 20 17:32:04.516071 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:04.516037 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgzwp\" (UniqueName: \"kubernetes.io/projected/1364a297-1675-430d-9fea-9cb25925923a-kube-api-access-lgzwp\") pod \"1364a297-1675-430d-9fea-9cb25925923a\" (UID: \"1364a297-1675-430d-9fea-9cb25925923a\") " Jul 20 17:32:04.518356 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:04.518332 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1364a297-1675-430d-9fea-9cb25925923a-kube-api-access-lgzwp" (OuterVolumeSpecName: "kube-api-access-lgzwp") pod "1364a297-1675-430d-9fea-9cb25925923a" (UID: "1364a297-1675-430d-9fea-9cb25925923a"). InnerVolumeSpecName "kube-api-access-lgzwp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:32:04.617257 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:04.617221 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgzwp\" (UniqueName: \"kubernetes.io/projected/1364a297-1675-430d-9fea-9cb25925923a-kube-api-access-lgzwp\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:32:04.656899 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:04.656853 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7269-140156648300880"] Jul 20 17:32:04.657529 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:04.657503 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1364a297-1675-430d-9fea-9cb25925923a" containerName="test-curl-7269-140156648352944" Jul 20 17:32:04.657529 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:04.657531 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1364a297-1675-430d-9fea-9cb25925923a" containerName="test-curl-7269-140156648352944" Jul 20 17:32:04.657747 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:04.657650 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1364a297-1675-430d-9fea-9cb25925923a" containerName="test-curl-7269-140156648352944" Jul 20 17:32:04.661044 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:04.661023 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7269-140156648300880"] Jul 20 17:32:04.661148 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:04.661101 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648300880" Jul 20 17:32:04.717812 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:04.717756 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbzdz\" (UniqueName: \"kubernetes.io/projected/6e4384f8-fe34-4fc9-99aa-29cc79cb725a-kube-api-access-pbzdz\") pod \"test-curl-7269-140156648300880\" (UID: \"6e4384f8-fe34-4fc9-99aa-29cc79cb725a\") " pod="opendatahub/test-curl-7269-140156648300880" Jul 20 17:32:04.819144 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:04.819107 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbzdz\" (UniqueName: \"kubernetes.io/projected/6e4384f8-fe34-4fc9-99aa-29cc79cb725a-kube-api-access-pbzdz\") pod \"test-curl-7269-140156648300880\" (UID: \"6e4384f8-fe34-4fc9-99aa-29cc79cb725a\") " pod="opendatahub/test-curl-7269-140156648300880" Jul 20 17:32:04.828623 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:04.828591 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbzdz\" (UniqueName: \"kubernetes.io/projected/6e4384f8-fe34-4fc9-99aa-29cc79cb725a-kube-api-access-pbzdz\") pod \"test-curl-7269-140156648300880\" (UID: \"6e4384f8-fe34-4fc9-99aa-29cc79cb725a\") " pod="opendatahub/test-curl-7269-140156648300880" Jul 20 17:32:04.970354 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:04.970275 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648300880" Jul 20 17:32:05.109392 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:05.109361 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7269-140156648300880"] Jul 20 17:32:05.111249 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:32:05.111222 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e4384f8_fe34_4fc9_99aa_29cc79cb725a.slice/crio-d89fd8eb9a1d71372d9b3092fe3a08cbc6d3420244867fa0d7a11f86310107ba WatchSource:0}: Error finding container d89fd8eb9a1d71372d9b3092fe3a08cbc6d3420244867fa0d7a11f86310107ba: Status 404 returned error can't find the container with id d89fd8eb9a1d71372d9b3092fe3a08cbc6d3420244867fa0d7a11f86310107ba Jul 20 17:32:05.294224 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:05.294129 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2fdd57683efd062884f89720dafb6e64d513c975163dcf8253e96dda5bfce634" Jul 20 17:32:05.294224 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:05.294161 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648352944" Jul 20 17:32:05.295401 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:05.295371 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7269-140156648300880" event={"ID":"6e4384f8-fe34-4fc9-99aa-29cc79cb725a","Type":"ContainerStarted","Data":"d89fd8eb9a1d71372d9b3092fe3a08cbc6d3420244867fa0d7a11f86310107ba"} Jul 20 17:32:05.304738 ip-10-0-130-18 kubenswrapper[2566]: E0720 17:32:05.304711 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7269-140156648352944\" is forbidden: User \"system:node:ip-10-0-130-18.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-18.ec2.internal' and this object" podUID="1364a297-1675-430d-9fea-9cb25925923a" pod="opendatahub/test-curl-7269-140156648352944" Jul 20 17:32:05.814041 ip-10-0-130-18 kubenswrapper[2566]: E0720 17:32:05.813995 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7269-140156648352944\" is forbidden: User \"system:node:ip-10-0-130-18.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-18.ec2.internal' and this object" podUID="1364a297-1675-430d-9fea-9cb25925923a" pod="opendatahub/test-curl-7269-140156648352944" Jul 20 17:32:05.815435 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:05.815413 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1364a297-1675-430d-9fea-9cb25925923a" path="/var/lib/kubelet/pods/1364a297-1675-430d-9fea-9cb25925923a/volumes" Jul 20 17:32:06.302999 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:06.302963 2566 generic.go:356] "Generic (PLEG): container finished" podID="6e4384f8-fe34-4fc9-99aa-29cc79cb725a" containerID="e5ec0296ec0c0931f550cf0b091a6d7ad2d3a74301298857a608bf09b4dde0a4" exitCode=0 Jul 20 17:32:06.303172 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:06.303044 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7269-140156648300880" event={"ID":"6e4384f8-fe34-4fc9-99aa-29cc79cb725a","Type":"ContainerDied","Data":"e5ec0296ec0c0931f550cf0b091a6d7ad2d3a74301298857a608bf09b4dde0a4"} Jul 20 17:32:07.448296 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:07.448265 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648300880" Jul 20 17:32:07.471273 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:07.471234 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7269-140156648300880_6e4384f8-fe34-4fc9-99aa-29cc79cb725a/test-curl-7269-140156648300880/0.log" Jul 20 17:32:07.497627 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:07.497590 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7269-140156648300880"] Jul 20 17:32:07.501783 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:07.501741 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7269-140156648300880"] Jul 20 17:32:07.541391 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:07.541361 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbzdz\" (UniqueName: \"kubernetes.io/projected/6e4384f8-fe34-4fc9-99aa-29cc79cb725a-kube-api-access-pbzdz\") pod \"6e4384f8-fe34-4fc9-99aa-29cc79cb725a\" (UID: \"6e4384f8-fe34-4fc9-99aa-29cc79cb725a\") " Jul 20 17:32:07.543650 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:07.543618 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e4384f8-fe34-4fc9-99aa-29cc79cb725a-kube-api-access-pbzdz" (OuterVolumeSpecName: "kube-api-access-pbzdz") pod "6e4384f8-fe34-4fc9-99aa-29cc79cb725a" (UID: "6e4384f8-fe34-4fc9-99aa-29cc79cb725a"). InnerVolumeSpecName "kube-api-access-pbzdz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:32:07.621418 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:07.621380 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7269-140156648300880"] Jul 20 17:32:07.621808 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:07.621792 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e4384f8-fe34-4fc9-99aa-29cc79cb725a" containerName="test-curl-7269-140156648300880" Jul 20 17:32:07.621808 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:07.621808 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e4384f8-fe34-4fc9-99aa-29cc79cb725a" containerName="test-curl-7269-140156648300880" Jul 20 17:32:07.621894 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:07.621889 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e4384f8-fe34-4fc9-99aa-29cc79cb725a" containerName="test-curl-7269-140156648300880" Jul 20 17:32:07.624836 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:07.624818 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648300880" Jul 20 17:32:07.630917 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:07.630884 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7269-140156648300880"] Jul 20 17:32:07.642079 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:07.642044 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v5xn\" (UniqueName: \"kubernetes.io/projected/ffd06b32-f26e-4c95-b524-993589246a5a-kube-api-access-5v5xn\") pod \"test-curl-7269-140156648300880\" (UID: \"ffd06b32-f26e-4c95-b524-993589246a5a\") " pod="opendatahub/test-curl-7269-140156648300880" Jul 20 17:32:07.642269 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:07.642097 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbzdz\" (UniqueName: \"kubernetes.io/projected/6e4384f8-fe34-4fc9-99aa-29cc79cb725a-kube-api-access-pbzdz\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:32:07.742938 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:07.742904 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v5xn\" (UniqueName: \"kubernetes.io/projected/ffd06b32-f26e-4c95-b524-993589246a5a-kube-api-access-5v5xn\") pod \"test-curl-7269-140156648300880\" (UID: \"ffd06b32-f26e-4c95-b524-993589246a5a\") " pod="opendatahub/test-curl-7269-140156648300880" Jul 20 17:32:07.751732 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:07.751712 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v5xn\" (UniqueName: \"kubernetes.io/projected/ffd06b32-f26e-4c95-b524-993589246a5a-kube-api-access-5v5xn\") pod \"test-curl-7269-140156648300880\" (UID: \"ffd06b32-f26e-4c95-b524-993589246a5a\") " pod="opendatahub/test-curl-7269-140156648300880" Jul 20 17:32:07.815109 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:07.815074 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e4384f8-fe34-4fc9-99aa-29cc79cb725a" path="/var/lib/kubelet/pods/6e4384f8-fe34-4fc9-99aa-29cc79cb725a/volumes" Jul 20 17:32:07.936029 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:07.935934 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648300880" Jul 20 17:32:08.071546 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:08.071469 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7269-140156648300880"] Jul 20 17:32:08.076460 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:32:08.076433 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podffd06b32_f26e_4c95_b524_993589246a5a.slice/crio-8b18b60ff26e612fc13348f53a2ab634659b86483ecafb05b51fd5466f1a13b7 WatchSource:0}: Error finding container 8b18b60ff26e612fc13348f53a2ab634659b86483ecafb05b51fd5466f1a13b7: Status 404 returned error can't find the container with id 8b18b60ff26e612fc13348f53a2ab634659b86483ecafb05b51fd5466f1a13b7 Jul 20 17:32:08.315739 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:08.315649 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648300880" Jul 20 17:32:08.315739 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:08.315712 2566 scope.go:117] "RemoveContainer" containerID="e5ec0296ec0c0931f550cf0b091a6d7ad2d3a74301298857a608bf09b4dde0a4" Jul 20 17:32:08.317057 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:08.317010 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7269-140156648300880" event={"ID":"ffd06b32-f26e-4c95-b524-993589246a5a","Type":"ContainerStarted","Data":"8b18b60ff26e612fc13348f53a2ab634659b86483ecafb05b51fd5466f1a13b7"} Jul 20 17:32:09.326484 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:09.326451 2566 generic.go:356] "Generic (PLEG): container finished" podID="ffd06b32-f26e-4c95-b524-993589246a5a" containerID="b7d2d1f213fd259a9126fc0914e4c8161e44d386faad46fb27f1cd0cbd0ff0e9" exitCode=0 Jul 20 17:32:09.326484 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:09.326485 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7269-140156648300880" event={"ID":"ffd06b32-f26e-4c95-b524-993589246a5a","Type":"ContainerDied","Data":"b7d2d1f213fd259a9126fc0914e4c8161e44d386faad46fb27f1cd0cbd0ff0e9"} Jul 20 17:32:10.474119 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:10.474091 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648300880" Jul 20 17:32:10.512649 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:10.512623 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7269-140156648300880_ffd06b32-f26e-4c95-b524-993589246a5a/test-curl-7269-140156648300880/0.log" Jul 20 17:32:10.555277 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:10.555243 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7269-140156648300880"] Jul 20 17:32:10.562327 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:10.562300 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7269-140156648300880"] Jul 20 17:32:10.569419 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:10.569394 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5v5xn\" (UniqueName: \"kubernetes.io/projected/ffd06b32-f26e-4c95-b524-993589246a5a-kube-api-access-5v5xn\") pod \"ffd06b32-f26e-4c95-b524-993589246a5a\" (UID: \"ffd06b32-f26e-4c95-b524-993589246a5a\") " Jul 20 17:32:10.571640 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:10.571611 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffd06b32-f26e-4c95-b524-993589246a5a-kube-api-access-5v5xn" (OuterVolumeSpecName: "kube-api-access-5v5xn") pod "ffd06b32-f26e-4c95-b524-993589246a5a" (UID: "ffd06b32-f26e-4c95-b524-993589246a5a"). InnerVolumeSpecName "kube-api-access-5v5xn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:32:10.671014 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:10.670907 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5v5xn\" (UniqueName: \"kubernetes.io/projected/ffd06b32-f26e-4c95-b524-993589246a5a-kube-api-access-5v5xn\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:32:10.690204 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:10.690167 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7269-140156648412192"] Jul 20 17:32:10.690622 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:10.690606 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffd06b32-f26e-4c95-b524-993589246a5a" containerName="test-curl-7269-140156648300880" Jul 20 17:32:10.690678 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:10.690624 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffd06b32-f26e-4c95-b524-993589246a5a" containerName="test-curl-7269-140156648300880" Jul 20 17:32:10.690718 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:10.690694 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffd06b32-f26e-4c95-b524-993589246a5a" containerName="test-curl-7269-140156648300880" Jul 20 17:32:10.693710 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:10.693694 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648412192" Jul 20 17:32:10.698708 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:10.698680 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7269-140156648412192"] Jul 20 17:32:10.771324 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:10.771286 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bfz7\" (UniqueName: \"kubernetes.io/projected/3eecc88b-1550-4376-bca4-fb0744eee3ef-kube-api-access-6bfz7\") pod \"test-curl-7269-140156648412192\" (UID: \"3eecc88b-1550-4376-bca4-fb0744eee3ef\") " pod="opendatahub/test-curl-7269-140156648412192" Jul 20 17:32:10.872745 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:10.872703 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bfz7\" (UniqueName: \"kubernetes.io/projected/3eecc88b-1550-4376-bca4-fb0744eee3ef-kube-api-access-6bfz7\") pod \"test-curl-7269-140156648412192\" (UID: \"3eecc88b-1550-4376-bca4-fb0744eee3ef\") " pod="opendatahub/test-curl-7269-140156648412192" Jul 20 17:32:10.881596 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:10.881560 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bfz7\" (UniqueName: \"kubernetes.io/projected/3eecc88b-1550-4376-bca4-fb0744eee3ef-kube-api-access-6bfz7\") pod \"test-curl-7269-140156648412192\" (UID: \"3eecc88b-1550-4376-bca4-fb0744eee3ef\") " pod="opendatahub/test-curl-7269-140156648412192" Jul 20 17:32:11.004947 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:11.004864 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648412192" Jul 20 17:32:11.139561 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:11.139531 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7269-140156648412192"] Jul 20 17:32:11.140238 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:32:11.140217 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3eecc88b_1550_4376_bca4_fb0744eee3ef.slice/crio-66522cd5057e70a4684ab41f93a0a1ed416630689389fd4359bc66b9fae37f08 WatchSource:0}: Error finding container 66522cd5057e70a4684ab41f93a0a1ed416630689389fd4359bc66b9fae37f08: Status 404 returned error can't find the container with id 66522cd5057e70a4684ab41f93a0a1ed416630689389fd4359bc66b9fae37f08 Jul 20 17:32:11.339618 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:11.339588 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648300880" Jul 20 17:32:11.339814 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:11.339626 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b18b60ff26e612fc13348f53a2ab634659b86483ecafb05b51fd5466f1a13b7" Jul 20 17:32:11.340956 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:11.340924 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7269-140156648412192" event={"ID":"3eecc88b-1550-4376-bca4-fb0744eee3ef","Type":"ContainerStarted","Data":"66522cd5057e70a4684ab41f93a0a1ed416630689389fd4359bc66b9fae37f08"} Jul 20 17:32:11.350284 ip-10-0-130-18 kubenswrapper[2566]: E0720 17:32:11.350256 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7269-140156648300880\" is forbidden: User \"system:node:ip-10-0-130-18.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-18.ec2.internal' and this object" podUID="ffd06b32-f26e-4c95-b524-993589246a5a" pod="opendatahub/test-curl-7269-140156648300880" Jul 20 17:32:11.815641 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:11.815557 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffd06b32-f26e-4c95-b524-993589246a5a" path="/var/lib/kubelet/pods/ffd06b32-f26e-4c95-b524-993589246a5a/volumes" Jul 20 17:32:12.348641 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:12.348608 2566 generic.go:356] "Generic (PLEG): container finished" podID="3eecc88b-1550-4376-bca4-fb0744eee3ef" containerID="baec85b1f0784f70c721d6d1fabc40b2db8d1c7eb172f2ab188e409cc6b3b9c9" exitCode=0 Jul 20 17:32:12.348858 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:12.348709 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7269-140156648412192" event={"ID":"3eecc88b-1550-4376-bca4-fb0744eee3ef","Type":"ContainerDied","Data":"baec85b1f0784f70c721d6d1fabc40b2db8d1c7eb172f2ab188e409cc6b3b9c9"} Jul 20 17:32:13.494288 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:13.494263 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648412192" Jul 20 17:32:13.516869 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:13.516840 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7269-140156648412192_3eecc88b-1550-4376-bca4-fb0744eee3ef/test-curl-7269-140156648412192/0.log" Jul 20 17:32:13.543101 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:13.543061 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7269-140156648412192"] Jul 20 17:32:13.545431 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:13.545403 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7269-140156648412192"] Jul 20 17:32:13.595379 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:13.595343 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bfz7\" (UniqueName: \"kubernetes.io/projected/3eecc88b-1550-4376-bca4-fb0744eee3ef-kube-api-access-6bfz7\") pod \"3eecc88b-1550-4376-bca4-fb0744eee3ef\" (UID: \"3eecc88b-1550-4376-bca4-fb0744eee3ef\") " Jul 20 17:32:13.597551 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:13.597525 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eecc88b-1550-4376-bca4-fb0744eee3ef-kube-api-access-6bfz7" (OuterVolumeSpecName: "kube-api-access-6bfz7") pod "3eecc88b-1550-4376-bca4-fb0744eee3ef" (UID: "3eecc88b-1550-4376-bca4-fb0744eee3ef"). InnerVolumeSpecName "kube-api-access-6bfz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:32:13.675298 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:13.675212 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7269-140156648412192"] Jul 20 17:32:13.675630 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:13.675616 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eecc88b-1550-4376-bca4-fb0744eee3ef" containerName="test-curl-7269-140156648412192" Jul 20 17:32:13.675677 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:13.675632 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eecc88b-1550-4376-bca4-fb0744eee3ef" containerName="test-curl-7269-140156648412192" Jul 20 17:32:13.675716 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:13.675707 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eecc88b-1550-4376-bca4-fb0744eee3ef" containerName="test-curl-7269-140156648412192" Jul 20 17:32:13.678757 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:13.678741 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648412192" Jul 20 17:32:13.689880 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:13.689852 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7269-140156648412192"] Jul 20 17:32:13.696334 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:13.696308 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85p7n\" (UniqueName: \"kubernetes.io/projected/5874163e-8585-4638-8b2b-9cafb4070561-kube-api-access-85p7n\") pod \"test-curl-7269-140156648412192\" (UID: \"5874163e-8585-4638-8b2b-9cafb4070561\") " pod="opendatahub/test-curl-7269-140156648412192" Jul 20 17:32:13.696522 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:13.696385 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bfz7\" (UniqueName: \"kubernetes.io/projected/3eecc88b-1550-4376-bca4-fb0744eee3ef-kube-api-access-6bfz7\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:32:13.797296 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:13.797253 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85p7n\" (UniqueName: \"kubernetes.io/projected/5874163e-8585-4638-8b2b-9cafb4070561-kube-api-access-85p7n\") pod \"test-curl-7269-140156648412192\" (UID: \"5874163e-8585-4638-8b2b-9cafb4070561\") " pod="opendatahub/test-curl-7269-140156648412192" Jul 20 17:32:13.806314 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:13.806283 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85p7n\" (UniqueName: \"kubernetes.io/projected/5874163e-8585-4638-8b2b-9cafb4070561-kube-api-access-85p7n\") pod \"test-curl-7269-140156648412192\" (UID: \"5874163e-8585-4638-8b2b-9cafb4070561\") " pod="opendatahub/test-curl-7269-140156648412192" Jul 20 17:32:13.817061 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:13.817035 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3eecc88b-1550-4376-bca4-fb0744eee3ef" path="/var/lib/kubelet/pods/3eecc88b-1550-4376-bca4-fb0744eee3ef/volumes" Jul 20 17:32:13.988349 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:13.988267 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648412192" Jul 20 17:32:14.127630 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:14.127597 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7269-140156648412192"] Jul 20 17:32:14.129312 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:32:14.129286 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5874163e_8585_4638_8b2b_9cafb4070561.slice/crio-6d0491fbe3d49c9dd49f47f29ed9930ad24733a0c65a66539e188398b6bd2177 WatchSource:0}: Error finding container 6d0491fbe3d49c9dd49f47f29ed9930ad24733a0c65a66539e188398b6bd2177: Status 404 returned error can't find the container with id 6d0491fbe3d49c9dd49f47f29ed9930ad24733a0c65a66539e188398b6bd2177 Jul 20 17:32:14.361455 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:14.361428 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648412192" Jul 20 17:32:14.361660 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:14.361489 2566 scope.go:117] "RemoveContainer" containerID="baec85b1f0784f70c721d6d1fabc40b2db8d1c7eb172f2ab188e409cc6b3b9c9" Jul 20 17:32:14.362857 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:14.362836 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7269-140156648412192" event={"ID":"5874163e-8585-4638-8b2b-9cafb4070561","Type":"ContainerStarted","Data":"6d0491fbe3d49c9dd49f47f29ed9930ad24733a0c65a66539e188398b6bd2177"} Jul 20 17:32:15.370959 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:15.370918 2566 generic.go:356] "Generic (PLEG): container finished" podID="5874163e-8585-4638-8b2b-9cafb4070561" containerID="3d8f9b23d4ffa0b8bc6a7297eab591e954e9ff8d74724342273ec77a0fd30a2e" exitCode=0 Jul 20 17:32:15.371364 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:15.371042 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7269-140156648412192" event={"ID":"5874163e-8585-4638-8b2b-9cafb4070561","Type":"ContainerDied","Data":"3d8f9b23d4ffa0b8bc6a7297eab591e954e9ff8d74724342273ec77a0fd30a2e"} Jul 20 17:32:16.511659 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:16.511634 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648412192" Jul 20 17:32:16.536361 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:16.536328 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7269-140156648412192_5874163e-8585-4638-8b2b-9cafb4070561/test-curl-7269-140156648412192/0.log" Jul 20 17:32:16.568841 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:16.568801 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7269-140156648412192"] Jul 20 17:32:16.569003 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:16.568864 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7269-140156648412192"] Jul 20 17:32:16.624641 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:16.624606 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85p7n\" (UniqueName: \"kubernetes.io/projected/5874163e-8585-4638-8b2b-9cafb4070561-kube-api-access-85p7n\") pod \"5874163e-8585-4638-8b2b-9cafb4070561\" (UID: \"5874163e-8585-4638-8b2b-9cafb4070561\") " Jul 20 17:32:16.626863 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:16.626841 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5874163e-8585-4638-8b2b-9cafb4070561-kube-api-access-85p7n" (OuterVolumeSpecName: "kube-api-access-85p7n") pod "5874163e-8585-4638-8b2b-9cafb4070561" (UID: "5874163e-8585-4638-8b2b-9cafb4070561"). InnerVolumeSpecName "kube-api-access-85p7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:32:16.725499 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:16.725458 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85p7n\" (UniqueName: \"kubernetes.io/projected/5874163e-8585-4638-8b2b-9cafb4070561-kube-api-access-85p7n\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:32:16.748470 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:16.748440 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f5995d-6w97w"] Jul 20 17:32:16.748986 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:16.748965 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5874163e-8585-4638-8b2b-9cafb4070561" containerName="test-curl-7269-140156648412192" Jul 20 17:32:16.748986 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:16.748984 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5874163e-8585-4638-8b2b-9cafb4070561" containerName="test-curl-7269-140156648412192" Jul 20 17:32:16.749137 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:16.749055 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5874163e-8585-4638-8b2b-9cafb4070561" containerName="test-curl-7269-140156648412192" Jul 20 17:32:16.752200 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:16.752186 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f5995d-6w97w" Jul 20 17:32:16.754339 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:16.754319 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5jc2q\"" Jul 20 17:32:16.759830 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:16.759802 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f5995d-6w97w"] Jul 20 17:32:16.826554 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:16.826513 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3835d23b-99b4-4359-887a-bf2b6c9a7656-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f5995d-6w97w\" (UID: \"3835d23b-99b4-4359-887a-bf2b6c9a7656\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f5995d-6w97w" Jul 20 17:32:16.927918 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:16.927832 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3835d23b-99b4-4359-887a-bf2b6c9a7656-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f5995d-6w97w\" (UID: \"3835d23b-99b4-4359-887a-bf2b6c9a7656\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f5995d-6w97w" Jul 20 17:32:16.928457 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:16.928439 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3835d23b-99b4-4359-887a-bf2b6c9a7656-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f5995d-6w97w\" (UID: \"3835d23b-99b4-4359-887a-bf2b6c9a7656\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f5995d-6w97w" Jul 20 17:32:17.064850 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:17.064814 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f5995d-6w97w" Jul 20 17:32:17.385632 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:17.385605 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7269-140156648412192" Jul 20 17:32:17.385829 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:17.385634 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d0491fbe3d49c9dd49f47f29ed9930ad24733a0c65a66539e188398b6bd2177" Jul 20 17:32:17.395867 ip-10-0-130-18 kubenswrapper[2566]: E0720 17:32:17.395836 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7269-140156648412192\" is forbidden: User \"system:node:ip-10-0-130-18.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-18.ec2.internal' and this object" podUID="5874163e-8585-4638-8b2b-9cafb4070561" pod="opendatahub/test-curl-7269-140156648412192" Jul 20 17:32:17.419569 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:17.419542 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f5995d-6w97w"] Jul 20 17:32:17.420933 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:32:17.420906 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3835d23b_99b4_4359_887a_bf2b6c9a7656.slice/crio-e5b63a6a7af6b66bc08beb0b6b62dee982a57b048d4e9cf96e8796cc5e8a5f12 WatchSource:0}: Error finding container e5b63a6a7af6b66bc08beb0b6b62dee982a57b048d4e9cf96e8796cc5e8a5f12: Status 404 returned error can't find the container with id e5b63a6a7af6b66bc08beb0b6b62dee982a57b048d4e9cf96e8796cc5e8a5f12 Jul 20 17:32:17.820382 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:17.820346 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5874163e-8585-4638-8b2b-9cafb4070561" path="/var/lib/kubelet/pods/5874163e-8585-4638-8b2b-9cafb4070561/volumes" Jul 20 17:32:18.393654 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:18.393561 2566 generic.go:356] "Generic (PLEG): container finished" podID="3835d23b-99b4-4359-887a-bf2b6c9a7656" containerID="d8e4e4741b8d1f8835f0f92f3055cfa6e12b0d4b3a0a0bcb40c0b5da4d86630b" exitCode=0 Jul 20 17:32:18.393654 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:18.393631 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f5995d-6w97w" event={"ID":"3835d23b-99b4-4359-887a-bf2b6c9a7656","Type":"ContainerDied","Data":"d8e4e4741b8d1f8835f0f92f3055cfa6e12b0d4b3a0a0bcb40c0b5da4d86630b"} Jul 20 17:32:18.393912 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:18.393680 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f5995d-6w97w" event={"ID":"3835d23b-99b4-4359-887a-bf2b6c9a7656","Type":"ContainerStarted","Data":"e5b63a6a7af6b66bc08beb0b6b62dee982a57b048d4e9cf96e8796cc5e8a5f12"} Jul 20 17:32:19.534006 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:19.533981 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f5995d-6w97w" Jul 20 17:32:19.652627 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:19.652531 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3835d23b-99b4-4359-887a-bf2b6c9a7656-maas-api-service-ca\") pod \"3835d23b-99b4-4359-887a-bf2b6c9a7656\" (UID: \"3835d23b-99b4-4359-887a-bf2b6c9a7656\") " Jul 20 17:32:19.652963 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:19.652939 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3835d23b-99b4-4359-887a-bf2b6c9a7656-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3835d23b-99b4-4359-887a-bf2b6c9a7656" (UID: "3835d23b-99b4-4359-887a-bf2b6c9a7656"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:32:19.753447 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:19.753403 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3835d23b-99b4-4359-887a-bf2b6c9a7656-maas-api-service-ca\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:32:20.405660 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:20.405631 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f5995d-6w97w" Jul 20 17:32:20.405842 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:20.405659 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f5995d-6w97w" event={"ID":"3835d23b-99b4-4359-887a-bf2b6c9a7656","Type":"ContainerDied","Data":"e5b63a6a7af6b66bc08beb0b6b62dee982a57b048d4e9cf96e8796cc5e8a5f12"} Jul 20 17:32:20.405842 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:20.405704 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e5b63a6a7af6b66bc08beb0b6b62dee982a57b048d4e9cf96e8796cc5e8a5f12" Jul 20 17:32:27.457840 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:27.457798 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f5995d-29742810-d8zk7"] Jul 20 17:32:27.463007 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:27.462982 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f5995d-29742810-d8zk7"] Jul 20 17:32:27.815394 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:27.815310 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="476890db-318f-46a2-8e06-cd78ffc0558f" path="/var/lib/kubelet/pods/476890db-318f-46a2-8e06-cd78ffc0558f/volumes" Jul 20 17:32:31.859899 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:31.859847 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f5995d-6w97w"] Jul 20 17:32:31.862379 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:31.862358 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f5995d-6w97w"] Jul 20 17:32:33.815303 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:33.815270 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3835d23b-99b4-4359-887a-bf2b6c9a7656" path="/var/lib/kubelet/pods/3835d23b-99b4-4359-887a-bf2b6c9a7656/volumes" Jul 20 17:32:38.856402 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:38.856360 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a24b97-w9pxh"] Jul 20 17:32:38.856855 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:38.856753 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3835d23b-99b4-4359-887a-bf2b6c9a7656" containerName="revoke-keys" Jul 20 17:32:38.856855 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:38.856784 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3835d23b-99b4-4359-887a-bf2b6c9a7656" containerName="revoke-keys" Jul 20 17:32:38.856943 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:38.856884 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3835d23b-99b4-4359-887a-bf2b6c9a7656" containerName="revoke-keys" Jul 20 17:32:38.868137 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:38.868108 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a24b97-w9pxh" Jul 20 17:32:38.869375 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:38.869345 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a24b97-w9pxh"] Jul 20 17:32:38.870265 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:38.870244 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5jc2q\"" Jul 20 17:32:39.031363 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:39.031326 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/128a0ddb-c686-410f-b017-723686e46604-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-a24b97-w9pxh\" (UID: \"128a0ddb-c686-410f-b017-723686e46604\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a24b97-w9pxh" Jul 20 17:32:39.132975 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:39.132874 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/128a0ddb-c686-410f-b017-723686e46604-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-a24b97-w9pxh\" (UID: \"128a0ddb-c686-410f-b017-723686e46604\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a24b97-w9pxh" Jul 20 17:32:39.133519 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:39.133502 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/128a0ddb-c686-410f-b017-723686e46604-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-a24b97-w9pxh\" (UID: \"128a0ddb-c686-410f-b017-723686e46604\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a24b97-w9pxh" Jul 20 17:32:39.179125 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:39.179090 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a24b97-w9pxh" Jul 20 17:32:39.319092 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:39.319062 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a24b97-w9pxh"] Jul 20 17:32:39.321000 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:32:39.320972 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod128a0ddb_c686_410f_b017_723686e46604.slice/crio-ab16686662422093f482bf23846e0e73538cb74ed28483f641f6654ca2b8ec19 WatchSource:0}: Error finding container ab16686662422093f482bf23846e0e73538cb74ed28483f641f6654ca2b8ec19: Status 404 returned error can't find the container with id ab16686662422093f482bf23846e0e73538cb74ed28483f641f6654ca2b8ec19 Jul 20 17:32:39.517708 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:39.517636 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a24b97-w9pxh" event={"ID":"128a0ddb-c686-410f-b017-723686e46604","Type":"ContainerStarted","Data":"ab16686662422093f482bf23846e0e73538cb74ed28483f641f6654ca2b8ec19"} Jul 20 17:32:40.524829 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:40.524760 2566 generic.go:356] "Generic (PLEG): container finished" podID="128a0ddb-c686-410f-b017-723686e46604" containerID="aec6d0bf6f5643017dde27b18e71e3f858a3ffd807717490e69b7c3442e3a956" exitCode=0 Jul 20 17:32:40.524829 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:40.524812 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a24b97-w9pxh" event={"ID":"128a0ddb-c686-410f-b017-723686e46604","Type":"ContainerDied","Data":"aec6d0bf6f5643017dde27b18e71e3f858a3ffd807717490e69b7c3442e3a956"} Jul 20 17:32:41.671962 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:41.671929 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a24b97-w9pxh" Jul 20 17:32:41.758661 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:41.758621 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/128a0ddb-c686-410f-b017-723686e46604-maas-api-service-ca\") pod \"128a0ddb-c686-410f-b017-723686e46604\" (UID: \"128a0ddb-c686-410f-b017-723686e46604\") " Jul 20 17:32:41.759032 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:41.759007 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/128a0ddb-c686-410f-b017-723686e46604-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "128a0ddb-c686-410f-b017-723686e46604" (UID: "128a0ddb-c686-410f-b017-723686e46604"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:32:41.859285 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:41.859256 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/128a0ddb-c686-410f-b017-723686e46604-maas-api-service-ca\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:32:42.537660 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:42.537627 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a24b97-w9pxh" Jul 20 17:32:42.537869 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:42.537667 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a24b97-w9pxh" event={"ID":"128a0ddb-c686-410f-b017-723686e46604","Type":"ContainerDied","Data":"ab16686662422093f482bf23846e0e73538cb74ed28483f641f6654ca2b8ec19"} Jul 20 17:32:42.537869 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:42.537725 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab16686662422093f482bf23846e0e73538cb74ed28483f641f6654ca2b8ec19" Jul 20 17:32:49.483026 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:49.482984 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a24b97-29742810-zdbgt"] Jul 20 17:32:49.487681 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:49.487655 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a24b97-29742810-zdbgt"] Jul 20 17:32:49.814962 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:49.814865 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3914a7ec-0305-4aba-83ba-e42c5f1714b5" path="/var/lib/kubelet/pods/3914a7ec-0305-4aba-83ba-e42c5f1714b5/volumes" Jul 20 17:32:53.944101 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:53.944060 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a24b97-w9pxh"] Jul 20 17:32:53.950400 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:53.950372 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a24b97-w9pxh"] Jul 20 17:32:55.814672 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:32:55.814634 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="128a0ddb-c686-410f-b017-723686e46604" path="/var/lib/kubelet/pods/128a0ddb-c686-410f-b017-723686e46604/volumes" Jul 20 17:33:00.542294 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:00.542259 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj"] Jul 20 17:33:00.542688 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:00.542481 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" podUID="7f0226da-5df5-46c3-8f02-6d692dd1f906" containerName="istio-proxy" containerID="cri-o://6652a9be57b3a12dcebb016f16f9800cdaca9bedea113701cd717862be9e7dd3" gracePeriod=30 Jul 20 17:33:01.213936 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:01.213895 2566 patch_prober.go:28] interesting pod/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 20 17:33:01.214160 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:01.213950 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" podUID="7f0226da-5df5-46c3-8f02-6d692dd1f906" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 17:33:01.220635 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:01.220608 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7f87b86957-zw899_9391dfd4-e0c4-47c6-a12e-a25c97292cb9/authorino/0.log" Jul 20 17:33:05.682807 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.682759 2566 generic.go:356] "Generic (PLEG): container finished" podID="7f0226da-5df5-46c3-8f02-6d692dd1f906" containerID="6652a9be57b3a12dcebb016f16f9800cdaca9bedea113701cd717862be9e7dd3" exitCode=0 Jul 20 17:33:05.683220 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.682824 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" event={"ID":"7f0226da-5df5-46c3-8f02-6d692dd1f906","Type":"ContainerDied","Data":"6652a9be57b3a12dcebb016f16f9800cdaca9bedea113701cd717862be9e7dd3"} Jul 20 17:33:05.683220 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.682938 2566 scope.go:117] "RemoveContainer" containerID="07908ebf410f7444b8839f7d2077f1d6981337622d0810ba64dc30696c36eaa5" Jul 20 17:33:05.805289 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.805218 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:33:05.900828 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.900759 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-envoy\") pod \"7f0226da-5df5-46c3-8f02-6d692dd1f906\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " Jul 20 17:33:05.901037 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.900846 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-podinfo\") pod \"7f0226da-5df5-46c3-8f02-6d692dd1f906\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " Jul 20 17:33:05.901037 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.900865 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-token\") pod \"7f0226da-5df5-46c3-8f02-6d692dd1f906\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " Jul 20 17:33:05.901037 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.900898 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khljw\" (UniqueName: \"kubernetes.io/projected/7f0226da-5df5-46c3-8f02-6d692dd1f906-kube-api-access-khljw\") pod \"7f0226da-5df5-46c3-8f02-6d692dd1f906\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " Jul 20 17:33:05.901037 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.900915 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f0226da-5df5-46c3-8f02-6d692dd1f906-istiod-ca-cert\") pod \"7f0226da-5df5-46c3-8f02-6d692dd1f906\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " Jul 20 17:33:05.901037 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.900929 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-workload-certs\") pod \"7f0226da-5df5-46c3-8f02-6d692dd1f906\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " Jul 20 17:33:05.901037 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.900967 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-data\") pod \"7f0226da-5df5-46c3-8f02-6d692dd1f906\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " Jul 20 17:33:05.901037 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.901007 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-workload-socket\") pod \"7f0226da-5df5-46c3-8f02-6d692dd1f906\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " Jul 20 17:33:05.901418 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.901103 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-credential-socket\") pod \"7f0226da-5df5-46c3-8f02-6d692dd1f906\" (UID: \"7f0226da-5df5-46c3-8f02-6d692dd1f906\") " Jul 20 17:33:05.901418 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.901366 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f0226da-5df5-46c3-8f02-6d692dd1f906-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7f0226da-5df5-46c3-8f02-6d692dd1f906" (UID: "7f0226da-5df5-46c3-8f02-6d692dd1f906"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:33:05.901533 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.901380 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7f0226da-5df5-46c3-8f02-6d692dd1f906" (UID: "7f0226da-5df5-46c3-8f02-6d692dd1f906"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:33:05.902005 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.901587 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f0226da-5df5-46c3-8f02-6d692dd1f906-istiod-ca-cert\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:33:05.902005 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.901611 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-workload-certs\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:33:05.902830 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.902132 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7f0226da-5df5-46c3-8f02-6d692dd1f906" (UID: "7f0226da-5df5-46c3-8f02-6d692dd1f906"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:33:05.902830 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.902152 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7f0226da-5df5-46c3-8f02-6d692dd1f906" (UID: "7f0226da-5df5-46c3-8f02-6d692dd1f906"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:33:05.902981 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.902871 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-data" (OuterVolumeSpecName: "istio-data") pod "7f0226da-5df5-46c3-8f02-6d692dd1f906" (UID: "7f0226da-5df5-46c3-8f02-6d692dd1f906"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:33:05.903435 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.903410 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7f0226da-5df5-46c3-8f02-6d692dd1f906" (UID: "7f0226da-5df5-46c3-8f02-6d692dd1f906"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 17:33:05.903532 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.903484 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f0226da-5df5-46c3-8f02-6d692dd1f906-kube-api-access-khljw" (OuterVolumeSpecName: "kube-api-access-khljw") pod "7f0226da-5df5-46c3-8f02-6d692dd1f906" (UID: "7f0226da-5df5-46c3-8f02-6d692dd1f906"). InnerVolumeSpecName "kube-api-access-khljw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:33:05.903532 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.903502 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-token" (OuterVolumeSpecName: "istio-token") pod "7f0226da-5df5-46c3-8f02-6d692dd1f906" (UID: "7f0226da-5df5-46c3-8f02-6d692dd1f906"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:33:05.903617 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:05.903548 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7f0226da-5df5-46c3-8f02-6d692dd1f906" (UID: "7f0226da-5df5-46c3-8f02-6d692dd1f906"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:33:06.002292 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:06.002249 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-credential-socket\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:33:06.002292 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:06.002288 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-envoy\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:33:06.002292 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:06.002298 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-podinfo\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:33:06.002589 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:06.002307 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-token\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:33:06.002589 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:06.002315 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khljw\" (UniqueName: \"kubernetes.io/projected/7f0226da-5df5-46c3-8f02-6d692dd1f906-kube-api-access-khljw\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:33:06.002589 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:06.002325 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-istio-data\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:33:06.002589 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:06.002334 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0226da-5df5-46c3-8f02-6d692dd1f906-workload-socket\") on node \"ip-10-0-130-18.ec2.internal\" DevicePath \"\"" Jul 20 17:33:06.690083 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:06.690051 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" Jul 20 17:33:06.690083 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:06.690076 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj" event={"ID":"7f0226da-5df5-46c3-8f02-6d692dd1f906","Type":"ContainerDied","Data":"74d007d72ad7ccfe3ea997c8109cc260a06c68cb12df1cef9c25c5899e9eaa5c"} Jul 20 17:33:06.690567 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:06.690143 2566 scope.go:117] "RemoveContainer" containerID="6652a9be57b3a12dcebb016f16f9800cdaca9bedea113701cd717862be9e7dd3" Jul 20 17:33:06.712539 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:06.712507 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj"] Jul 20 17:33:06.714995 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:06.714971 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-a24b97-openshift-default-5957c6bd79-28fsj"] Jul 20 17:33:07.503449 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:07.503419 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76b8cd99f7-z97kn_b7009ce5-2b66-46ec-8d78-648659de2682/copy-manifests/0.log" Jul 20 17:33:07.516708 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:07.516669 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76b8cd99f7-z97kn_b7009ce5-2b66-46ec-8d78-648659de2682/manager/0.log" Jul 20 17:33:07.815836 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:07.815732 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f0226da-5df5-46c3-8f02-6d692dd1f906" path="/var/lib/kubelet/pods/7f0226da-5df5-46c3-8f02-6d692dd1f906/volumes" Jul 20 17:33:07.994048 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:07.994016 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29742780-9dkm2_87d40f2f-c4a2-4b93-a215-59e1b5cc5809/cleanup/0.log" Jul 20 17:33:08.105569 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:08.105535 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29742795-n7nfv_ce967b0d-a6e3-4019-aca6-515a738cb73f/cleanup/0.log" Jul 20 17:33:08.221441 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:08.221412 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29742810-xsccz_b71db8d2-4741-4662-8b90-07b59710dbf3/cleanup/0.log" Jul 20 17:33:08.338164 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:08.338132 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-7kmd4_0f7cddb2-d9a8-4822-bd10-cb86784a5a45/postgres/0.log" Jul 20 17:33:09.123604 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:09.123576 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2_3e48ee72-a253-47ec-99ce-0da8e9bfb190/util/0.log" Jul 20 17:33:09.132149 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:09.132123 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2_3e48ee72-a253-47ec-99ce-0da8e9bfb190/pull/0.log" Jul 20 17:33:09.140221 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:09.140199 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2_3e48ee72-a253-47ec-99ce-0da8e9bfb190/extract/0.log" Jul 20 17:33:09.249138 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:09.249107 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq_cb82ff48-a85d-4311-95df-8eb56de4c627/util/0.log" Jul 20 17:33:09.256326 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:09.256299 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq_cb82ff48-a85d-4311-95df-8eb56de4c627/pull/0.log" Jul 20 17:33:09.262961 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:09.262932 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq_cb82ff48-a85d-4311-95df-8eb56de4c627/extract/0.log" Jul 20 17:33:09.375679 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:09.375600 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4_80e2fc4c-694a-4fde-b001-c75bbbfdad92/extract/0.log" Jul 20 17:33:09.382602 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:09.382578 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4_80e2fc4c-694a-4fde-b001-c75bbbfdad92/util/0.log" Jul 20 17:33:09.389606 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:09.389582 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4_80e2fc4c-694a-4fde-b001-c75bbbfdad92/pull/0.log" Jul 20 17:33:09.513109 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:09.513079 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq_48dcbfe7-9bc6-4174-8980-befb394c0923/extract/0.log" Jul 20 17:33:09.519924 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:09.519893 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq_48dcbfe7-9bc6-4174-8980-befb394c0923/util/0.log" Jul 20 17:33:09.526763 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:09.526743 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq_48dcbfe7-9bc6-4174-8980-befb394c0923/pull/0.log" Jul 20 17:33:09.648634 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:09.648552 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7f87b86957-zw899_9391dfd4-e0c4-47c6-a12e-a25c97292cb9/authorino/0.log" Jul 20 17:33:11.192089 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:11.192024 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-769f9dd78c-k2r9q_6748e8be-7893-4fc4-9b26-31ce6f90f03d/kube-auth-proxy/0.log" Jul 20 17:33:12.158821 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:12.158755 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-4vf6s_bc661ae8-5399-487b-ab29-84665fa72f41/main/0.log" Jul 20 17:33:12.497699 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:12.497606 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-nqs24_5699cd6e-e484-48ab-adb1-08339abca632/main/0.log" Jul 20 17:33:17.949322 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:17.949288 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z68mp/must-gather-njvhn"] Jul 20 17:33:17.949702 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:17.949675 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f0226da-5df5-46c3-8f02-6d692dd1f906" containerName="istio-proxy" Jul 20 17:33:17.949702 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:17.949687 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f0226da-5df5-46c3-8f02-6d692dd1f906" containerName="istio-proxy" Jul 20 17:33:17.949702 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:17.949699 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="128a0ddb-c686-410f-b017-723686e46604" containerName="revoke-keys" Jul 20 17:33:17.949825 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:17.949705 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="128a0ddb-c686-410f-b017-723686e46604" containerName="revoke-keys" Jul 20 17:33:17.949825 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:17.949718 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f0226da-5df5-46c3-8f02-6d692dd1f906" containerName="istio-proxy" Jul 20 17:33:17.949825 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:17.949724 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f0226da-5df5-46c3-8f02-6d692dd1f906" containerName="istio-proxy" Jul 20 17:33:17.949825 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:17.949810 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f0226da-5df5-46c3-8f02-6d692dd1f906" containerName="istio-proxy" Jul 20 17:33:17.949825 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:17.949821 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="128a0ddb-c686-410f-b017-723686e46604" containerName="revoke-keys" Jul 20 17:33:17.949978 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:17.949936 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f0226da-5df5-46c3-8f02-6d692dd1f906" containerName="istio-proxy" Jul 20 17:33:17.953066 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:17.953047 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z68mp/must-gather-njvhn" Jul 20 17:33:17.955571 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:17.955551 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-z68mp\"/\"default-dockercfg-58xlt\"" Jul 20 17:33:17.956069 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:17.956054 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z68mp\"/\"openshift-service-ca.crt\"" Jul 20 17:33:17.956183 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:17.956168 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z68mp\"/\"kube-root-ca.crt\"" Jul 20 17:33:17.967083 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:17.967056 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z68mp/must-gather-njvhn"] Jul 20 17:33:18.113070 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:18.113031 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2dcf094d-2632-4cf1-8b8a-998f8b1f64a3-must-gather-output\") pod \"must-gather-njvhn\" (UID: \"2dcf094d-2632-4cf1-8b8a-998f8b1f64a3\") " pod="openshift-must-gather-z68mp/must-gather-njvhn" Jul 20 17:33:18.113245 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:18.113083 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f225f\" (UniqueName: \"kubernetes.io/projected/2dcf094d-2632-4cf1-8b8a-998f8b1f64a3-kube-api-access-f225f\") pod \"must-gather-njvhn\" (UID: \"2dcf094d-2632-4cf1-8b8a-998f8b1f64a3\") " pod="openshift-must-gather-z68mp/must-gather-njvhn" Jul 20 17:33:18.214376 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:18.214290 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2dcf094d-2632-4cf1-8b8a-998f8b1f64a3-must-gather-output\") pod \"must-gather-njvhn\" (UID: \"2dcf094d-2632-4cf1-8b8a-998f8b1f64a3\") " pod="openshift-must-gather-z68mp/must-gather-njvhn" Jul 20 17:33:18.214376 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:18.214344 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f225f\" (UniqueName: \"kubernetes.io/projected/2dcf094d-2632-4cf1-8b8a-998f8b1f64a3-kube-api-access-f225f\") pod \"must-gather-njvhn\" (UID: \"2dcf094d-2632-4cf1-8b8a-998f8b1f64a3\") " pod="openshift-must-gather-z68mp/must-gather-njvhn" Jul 20 17:33:18.214673 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:18.214655 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2dcf094d-2632-4cf1-8b8a-998f8b1f64a3-must-gather-output\") pod \"must-gather-njvhn\" (UID: \"2dcf094d-2632-4cf1-8b8a-998f8b1f64a3\") " pod="openshift-must-gather-z68mp/must-gather-njvhn" Jul 20 17:33:18.225805 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:18.225762 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f225f\" (UniqueName: \"kubernetes.io/projected/2dcf094d-2632-4cf1-8b8a-998f8b1f64a3-kube-api-access-f225f\") pod \"must-gather-njvhn\" (UID: \"2dcf094d-2632-4cf1-8b8a-998f8b1f64a3\") " pod="openshift-must-gather-z68mp/must-gather-njvhn" Jul 20 17:33:18.262658 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:18.262627 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z68mp/must-gather-njvhn" Jul 20 17:33:18.401466 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:18.401441 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z68mp/must-gather-njvhn"] Jul 20 17:33:18.404488 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:33:18.404460 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2dcf094d_2632_4cf1_8b8a_998f8b1f64a3.slice/crio-48daccb62a09135c9cd9212a1bace03c7b170f248dbf2c2bf82a096b2314a295 WatchSource:0}: Error finding container 48daccb62a09135c9cd9212a1bace03c7b170f248dbf2c2bf82a096b2314a295: Status 404 returned error can't find the container with id 48daccb62a09135c9cd9212a1bace03c7b170f248dbf2c2bf82a096b2314a295 Jul 20 17:33:18.761431 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:18.761389 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z68mp/must-gather-njvhn" event={"ID":"2dcf094d-2632-4cf1-8b8a-998f8b1f64a3","Type":"ContainerStarted","Data":"48daccb62a09135c9cd9212a1bace03c7b170f248dbf2c2bf82a096b2314a295"} Jul 20 17:33:19.769108 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:19.769057 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z68mp/must-gather-njvhn" event={"ID":"2dcf094d-2632-4cf1-8b8a-998f8b1f64a3","Type":"ContainerStarted","Data":"f9a160853ed32cfcff4213caa2cf5be699759a1096617d9de60c49ae3d6133d8"} Jul 20 17:33:19.769108 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:19.769113 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z68mp/must-gather-njvhn" event={"ID":"2dcf094d-2632-4cf1-8b8a-998f8b1f64a3","Type":"ContainerStarted","Data":"5b4547ec51a6ce4b2de7255a32a05bc5cf2c6090234b1990eb2f8d8a846c8564"} Jul 20 17:33:19.790186 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:19.790127 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-z68mp/must-gather-njvhn" podStartSLOduration=1.947095822 podStartE2EDuration="2.790110011s" totalImagesPullingTime="843.014189ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:33:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:33:19.785622493 +0000 UTC m=+2864.571621572" watchObservedRunningTime="2026-07-20 17:33:19.790110011 +0000 UTC m=+2864.576109089" Jul 20 17:33:20.962346 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:20.962316 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-t2zqd_808cf239-1e7e-484f-89f2-e8e9ad673366/global-pull-secret-syncer/0.log" Jul 20 17:33:21.062852 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:21.062822 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vsvts_e820f776-0544-43e3-a7f3-de0235215c36/konnectivity-agent/0.log" Jul 20 17:33:21.145371 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:21.145324 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-18.ec2.internal_e496a6e153fcce1aa86c117a388fbbcb/haproxy/0.log" Jul 20 17:33:25.672845 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:25.672802 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2_3e48ee72-a253-47ec-99ce-0da8e9bfb190/extract/0.log" Jul 20 17:33:25.697639 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:25.697597 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2_3e48ee72-a253-47ec-99ce-0da8e9bfb190/util/0.log" Jul 20 17:33:25.724928 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:25.724820 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vbgz2_3e48ee72-a253-47ec-99ce-0da8e9bfb190/pull/0.log" Jul 20 17:33:25.754416 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:25.754382 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq_cb82ff48-a85d-4311-95df-8eb56de4c627/extract/0.log" Jul 20 17:33:25.779232 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:25.779176 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq_cb82ff48-a85d-4311-95df-8eb56de4c627/util/0.log" Jul 20 17:33:25.806249 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:25.806218 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r4wnq_cb82ff48-a85d-4311-95df-8eb56de4c627/pull/0.log" Jul 20 17:33:25.857090 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:25.857018 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4_80e2fc4c-694a-4fde-b001-c75bbbfdad92/extract/0.log" Jul 20 17:33:25.897810 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:25.897689 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4_80e2fc4c-694a-4fde-b001-c75bbbfdad92/util/0.log" Jul 20 17:33:25.923787 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:25.923660 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7357hx4_80e2fc4c-694a-4fde-b001-c75bbbfdad92/pull/0.log" Jul 20 17:33:25.952967 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:25.952893 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq_48dcbfe7-9bc6-4174-8980-befb394c0923/extract/0.log" Jul 20 17:33:25.985808 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:25.985724 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq_48dcbfe7-9bc6-4174-8980-befb394c0923/util/0.log" Jul 20 17:33:26.008597 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:26.008571 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qm4cq_48dcbfe7-9bc6-4174-8980-befb394c0923/pull/0.log" Jul 20 17:33:26.467837 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:26.467761 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7f87b86957-zw899_9391dfd4-e0c4-47c6-a12e-a25c97292cb9/authorino/0.log" Jul 20 17:33:28.492293 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:28.492267 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a5fd763-c746-41c4-988d-894a46a5e08b/alertmanager/0.log" Jul 20 17:33:28.541196 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:28.541147 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a5fd763-c746-41c4-988d-894a46a5e08b/config-reloader/0.log" Jul 20 17:33:28.567123 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:28.567095 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a5fd763-c746-41c4-988d-894a46a5e08b/kube-rbac-proxy-web/0.log" Jul 20 17:33:28.597230 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:28.597190 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a5fd763-c746-41c4-988d-894a46a5e08b/kube-rbac-proxy/0.log" Jul 20 17:33:28.627668 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:28.627617 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a5fd763-c746-41c4-988d-894a46a5e08b/kube-rbac-proxy-metric/0.log" Jul 20 17:33:28.660452 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:28.660413 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a5fd763-c746-41c4-988d-894a46a5e08b/prom-label-proxy/0.log" Jul 20 17:33:28.687472 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:28.687429 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a5fd763-c746-41c4-988d-894a46a5e08b/init-config-reloader/0.log" Jul 20 17:33:28.768200 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:28.768113 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-n8f49_125e8fb6-3cf7-4bf6-ba49-b02a6f580e44/kube-state-metrics/0.log" Jul 20 17:33:28.796864 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:28.796824 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-n8f49_125e8fb6-3cf7-4bf6-ba49-b02a6f580e44/kube-rbac-proxy-main/0.log" Jul 20 17:33:28.824746 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:28.824702 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-n8f49_125e8fb6-3cf7-4bf6-ba49-b02a6f580e44/kube-rbac-proxy-self/0.log" Jul 20 17:33:28.866993 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:28.866928 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-79fd676bf4-5kjr6_86854af6-6165-48b9-8ef0-e8b55a8e52aa/metrics-server/0.log" Jul 20 17:33:28.896799 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:28.896739 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-xpxdv_595b12c7-aa06-4bf8-96dd-34c503efa515/monitoring-plugin/0.log" Jul 20 17:33:29.008857 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:29.008824 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jr2tv_e2c1840a-8c58-4136-8479-c700fb9ccd2c/node-exporter/0.log" Jul 20 17:33:29.032012 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:29.031933 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jr2tv_e2c1840a-8c58-4136-8479-c700fb9ccd2c/kube-rbac-proxy/0.log" Jul 20 17:33:29.065031 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:29.064997 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jr2tv_e2c1840a-8c58-4136-8479-c700fb9ccd2c/init-textfile/0.log" Jul 20 17:33:29.175756 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:29.175720 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-gkfd9_6c716602-e8f7-4a96-ba67-b64a6126a0a2/kube-rbac-proxy-main/0.log" Jul 20 17:33:29.202849 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:29.202803 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-gkfd9_6c716602-e8f7-4a96-ba67-b64a6126a0a2/kube-rbac-proxy-self/0.log" Jul 20 17:33:29.227827 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:29.227766 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-gkfd9_6c716602-e8f7-4a96-ba67-b64a6126a0a2/openshift-state-metrics/0.log" Jul 20 17:33:29.446085 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:29.446051 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-777dm_19715257-abd0-4174-8253-48fecc5b79a9/prometheus-operator/0.log" Jul 20 17:33:29.467113 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:29.467084 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-777dm_19715257-abd0-4174-8253-48fecc5b79a9/kube-rbac-proxy/0.log" Jul 20 17:33:29.494458 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:29.494423 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-zrdtm_946f07eb-462e-48d7-a469-692c829dfabf/prometheus-operator-admission-webhook/0.log" Jul 20 17:33:29.523388 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:29.523357 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-86f5554fbd-88fdq_8196778f-460c-4050-80d5-950ec200e832/telemeter-client/0.log" Jul 20 17:33:29.548687 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:29.548623 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-86f5554fbd-88fdq_8196778f-460c-4050-80d5-950ec200e832/reload/0.log" Jul 20 17:33:29.578156 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:29.578126 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-86f5554fbd-88fdq_8196778f-460c-4050-80d5-950ec200e832/kube-rbac-proxy/0.log" Jul 20 17:33:29.610799 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:29.610744 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fbb45cdcd-dz559_8d7b1577-accd-455c-8511-9198592e5c51/thanos-query/0.log" Jul 20 17:33:29.633511 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:29.633471 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fbb45cdcd-dz559_8d7b1577-accd-455c-8511-9198592e5c51/kube-rbac-proxy-web/0.log" Jul 20 17:33:29.655868 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:29.655739 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fbb45cdcd-dz559_8d7b1577-accd-455c-8511-9198592e5c51/kube-rbac-proxy/0.log" Jul 20 17:33:29.680239 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:29.680203 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fbb45cdcd-dz559_8d7b1577-accd-455c-8511-9198592e5c51/prom-label-proxy/0.log" Jul 20 17:33:29.702218 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:29.702187 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fbb45cdcd-dz559_8d7b1577-accd-455c-8511-9198592e5c51/kube-rbac-proxy-rules/0.log" Jul 20 17:33:29.729087 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:29.729053 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fbb45cdcd-dz559_8d7b1577-accd-455c-8511-9198592e5c51/kube-rbac-proxy-metrics/0.log" Jul 20 17:33:31.544368 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.544324 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg"] Jul 20 17:33:31.553954 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.553916 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:31.556238 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.556207 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg"] Jul 20 17:33:31.684873 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.684825 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f-lib-modules\") pod \"perf-node-gather-daemonset-6qcsg\" (UID: \"a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:31.685070 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.684903 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f-sys\") pod \"perf-node-gather-daemonset-6qcsg\" (UID: \"a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:31.685070 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.684979 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f-podres\") pod \"perf-node-gather-daemonset-6qcsg\" (UID: \"a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:31.685070 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.685003 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcnxf\" (UniqueName: \"kubernetes.io/projected/a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f-kube-api-access-vcnxf\") pod \"perf-node-gather-daemonset-6qcsg\" (UID: \"a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:31.685224 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.685073 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f-proc\") pod \"perf-node-gather-daemonset-6qcsg\" (UID: \"a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:31.785640 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.785601 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f-podres\") pod \"perf-node-gather-daemonset-6qcsg\" (UID: \"a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:31.785640 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.785643 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcnxf\" (UniqueName: \"kubernetes.io/projected/a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f-kube-api-access-vcnxf\") pod \"perf-node-gather-daemonset-6qcsg\" (UID: \"a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:31.785941 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.785696 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f-proc\") pod \"perf-node-gather-daemonset-6qcsg\" (UID: \"a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:31.785941 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.785745 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f-lib-modules\") pod \"perf-node-gather-daemonset-6qcsg\" (UID: \"a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:31.785941 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.785812 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f-sys\") pod \"perf-node-gather-daemonset-6qcsg\" (UID: \"a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:31.785941 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.785841 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f-proc\") pod \"perf-node-gather-daemonset-6qcsg\" (UID: \"a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:31.785941 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.785842 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f-podres\") pod \"perf-node-gather-daemonset-6qcsg\" (UID: \"a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:31.785941 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.785896 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f-sys\") pod \"perf-node-gather-daemonset-6qcsg\" (UID: \"a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:31.786225 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.785944 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f-lib-modules\") pod \"perf-node-gather-daemonset-6qcsg\" (UID: \"a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:31.794366 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.794337 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcnxf\" (UniqueName: \"kubernetes.io/projected/a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f-kube-api-access-vcnxf\") pod \"perf-node-gather-daemonset-6qcsg\" (UID: \"a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:31.870110 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:31.870053 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:32.084033 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:32.084000 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg"] Jul 20 17:33:32.086419 ip-10-0-130-18 kubenswrapper[2566]: W0720 17:33:32.086369 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda550b3f9_c5a3_4bc5_8f0b_ce89dcb62b8f.slice/crio-38034bddf1d1b8bd0d25020e919013fa3fc54ba51056817f6bf5cff949bf73cc WatchSource:0}: Error finding container 38034bddf1d1b8bd0d25020e919013fa3fc54ba51056817f6bf5cff949bf73cc: Status 404 returned error can't find the container with id 38034bddf1d1b8bd0d25020e919013fa3fc54ba51056817f6bf5cff949bf73cc Jul 20 17:33:32.100746 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:32.100716 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5c8b5fd8c9-fsrqc_f267b375-354c-444b-8484-8465c41f89d0/console/0.log" Jul 20 17:33:32.894027 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:32.893985 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" event={"ID":"a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f","Type":"ContainerStarted","Data":"6ae166d47477af6099d96878d576fa2526c0620f10a69b1271d13410dfd908ef"} Jul 20 17:33:32.894525 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:32.894056 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:32.894525 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:32.894082 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" event={"ID":"a550b3f9-c5a3-4bc5-8f0b-ce89dcb62b8f","Type":"ContainerStarted","Data":"38034bddf1d1b8bd0d25020e919013fa3fc54ba51056817f6bf5cff949bf73cc"} Jul 20 17:33:32.910628 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:32.910561 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" podStartSLOduration=1.910541043 podStartE2EDuration="1.910541043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 17:33:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:33:32.908824341 +0000 UTC m=+2877.694823420" watchObservedRunningTime="2026-07-20 17:33:32.910541043 +0000 UTC m=+2877.696540123" Jul 20 17:33:33.812725 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:33.812697 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dwlfm_1014a924-f279-4b0f-b362-4f39c0c03be0/dns/0.log" Jul 20 17:33:33.837206 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:33.837177 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dwlfm_1014a924-f279-4b0f-b362-4f39c0c03be0/kube-rbac-proxy/0.log" Jul 20 17:33:33.980128 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:33.980099 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-88g67_507c01a0-2197-4761-b183-58de663742c4/dns-node-resolver/0.log" Jul 20 17:33:34.614609 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:34.614573 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ps2lf_a6b9b3cd-b414-46f7-9814-6b334f384e70/node-ca/0.log" Jul 20 17:33:35.763143 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:35.763099 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-769f9dd78c-k2r9q_6748e8be-7893-4fc4-9b26-31ce6f90f03d/kube-auth-proxy/0.log" Jul 20 17:33:36.620124 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:36.620093 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2xvv9_5b354580-bc22-4283-bbff-5e9e87fc026a/serve-healthcheck-canary/0.log" Jul 20 17:33:37.255056 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:37.255027 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tdgwh_98ea33d6-9c67-4e13-842a-caffa292568c/kube-rbac-proxy/0.log" Jul 20 17:33:37.281211 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:37.281183 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tdgwh_98ea33d6-9c67-4e13-842a-caffa292568c/exporter/0.log" Jul 20 17:33:37.309271 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:37.309233 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tdgwh_98ea33d6-9c67-4e13-842a-caffa292568c/extractor/0.log" Jul 20 17:33:38.911364 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:38.911335 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-6qcsg" Jul 20 17:33:40.399268 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:40.399177 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76b8cd99f7-z97kn_b7009ce5-2b66-46ec-8d78-648659de2682/manager/0.log" Jul 20 17:33:40.419837 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:40.419809 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76b8cd99f7-z97kn_b7009ce5-2b66-46ec-8d78-648659de2682/copy-manifests/0.log" Jul 20 17:33:41.921221 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:41.921192 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-86568699db-chx9m_0744defc-d845-428d-b757-60f6204a9d57/manager/0.log" Jul 20 17:33:41.976517 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:41.976446 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-pz67k_1a756eb7-f252-4777-806b-ec4162928819/openshift-lws-operator/0.log" Jul 20 17:33:42.270632 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:42.270606 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-769f9dd78c-k2r9q_6748e8be-7893-4fc4-9b26-31ce6f90f03d/kube-auth-proxy/0.log" Jul 20 17:33:43.820424 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:43.820339 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-769f9dd78c-k2r9q_6748e8be-7893-4fc4-9b26-31ce6f90f03d/kube-auth-proxy/0.log" Jul 20 17:33:48.409336 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:48.409307 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-27ncp_81f36e5a-cb50-4407-820e-506a836f18dc/kube-multus-additional-cni-plugins/0.log" Jul 20 17:33:48.449198 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:48.449168 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-27ncp_81f36e5a-cb50-4407-820e-506a836f18dc/egress-router-binary-copy/0.log" Jul 20 17:33:48.494021 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:48.493988 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-27ncp_81f36e5a-cb50-4407-820e-506a836f18dc/cni-plugins/0.log" Jul 20 17:33:48.533718 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:48.533640 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-27ncp_81f36e5a-cb50-4407-820e-506a836f18dc/bond-cni-plugin/0.log" Jul 20 17:33:48.577660 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:48.577630 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-27ncp_81f36e5a-cb50-4407-820e-506a836f18dc/routeoverride-cni/0.log" Jul 20 17:33:48.611544 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:48.611513 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-27ncp_81f36e5a-cb50-4407-820e-506a836f18dc/whereabouts-cni-bincopy/0.log" Jul 20 17:33:48.638191 ip-10-0-130-18 kubenswrapper[2566]: I0720 17:33:48.638161 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-27ncp_81f36e5a-cb50-4407-820e-506a836f18dc/whereabouts-cni/0.log"