Jul 10 18:58:34.933332 ip-10-0-131-229 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 10 18:58:34.933341 ip-10-0-131-229 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 10 18:58:34.933349 ip-10-0-131-229 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 10 18:58:34.933586 ip-10-0-131-229 systemd[1]: Failed to start Kubernetes Kubelet. Jul 10 18:58:45.182617 ip-10-0-131-229 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 10 18:58:45.182640 ip-10-0-131-229 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot f1f1734f779245b4a3d258856af6e17a -- Jul 10 19:00:18.333406 ip-10-0-131-229 systemd[1]: Starting Kubernetes Kubelet... Jul 10 19:00:18.805593 ip-10-0-131-229 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 19:00:18.805593 ip-10-0-131-229 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 19:00:18.805593 ip-10-0-131-229 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 19:00:18.805593 ip-10-0-131-229 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 19:00:18.805593 ip-10-0-131-229 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 19:00:18.805593 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.794907 2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 19:00:18.805593 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802034 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 19:00:18.805593 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802049 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 19:00:18.805593 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802055 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 19:00:18.805593 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802060 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 19:00:18.805593 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802063 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 19:00:18.805593 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802067 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 19:00:18.805593 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802070 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802072 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802075 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802078 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802081 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802084 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802086 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802089 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802091 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802094 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802113 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802116 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802120 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802129 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802132 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802135 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802138 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802140 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802143 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802145 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 19:00:18.867835 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802147 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802150 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802153 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802155 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802158 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802160 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802163 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802167 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802170 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802173 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802175 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802178 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802181 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802183 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802186 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802188 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802191 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802195 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802197 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802200 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 19:00:18.870274 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802202 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 19:00:18.869787 ip-10-0-131-229 systemd[1]: Started Kubernetes Kubelet. Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802205 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802207 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802210 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802213 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802215 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802218 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802223 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802226 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802229 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802232 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802236 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802239 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802241 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802244 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802246 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802250 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802253 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802255 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 19:00:19.063028 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802258 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802260 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802262 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802265 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802267 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802270 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802272 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802275 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802277 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802280 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802282 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802285 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802288 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802290 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802293 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802296 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802299 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802303 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802307 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 19:00:19.082574 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802310 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802313 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802316 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802318 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802320 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802323 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802325 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802328 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802330 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802332 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802335 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802337 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802340 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802342 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.802344 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803839 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803847 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803851 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803854 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803857 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 19:00:19.083415 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803860 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803863 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803865 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803868 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803871 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803874 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803877 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803879 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803882 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803885 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803887 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803890 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803894 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803897 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803899 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803903 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803907 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803910 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803913 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 19:00:19.084211 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803916 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803919 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803923 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803926 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803929 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803931 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803934 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803936 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803939 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803942 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803946 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803948 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803951 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803954 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803957 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803959 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803964 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803967 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803970 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 19:00:19.084906 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803973 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803976 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803979 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803982 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803984 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803987 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803992 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803995 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.803998 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804000 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804003 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804006 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804008 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804011 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804013 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804017 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804020 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804022 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804025 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 19:00:19.085571 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804027 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804030 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804033 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804037 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804040 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804042 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804045 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804047 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804050 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804052 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804055 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804057 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804060 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804062 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804065 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804067 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804070 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804072 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804075 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804078 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 19:00:19.086189 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804081 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804084 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804086 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804089 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804091 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804094 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804116 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804119 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804122 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804125 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804128 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804130 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804133 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804136 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804139 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804142 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804145 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804535 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804540 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 19:00:19.086794 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804543 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804547 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804551 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804555 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804559 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804562 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804565 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804568 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804570 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804573 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804575 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804579 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804583 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804586 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804590 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804592 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804595 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804598 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804600 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 19:00:19.087515 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804603 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804605 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804608 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804610 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804613 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804615 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804618 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804621 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804623 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804627 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804629 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804632 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804634 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804645 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804648 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804651 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804654 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804657 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804659 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804662 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 19:00:19.088061 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804664 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804667 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804669 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804671 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804674 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804678 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804681 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804683 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804689 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804692 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804694 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804697 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804699 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804702 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804704 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804707 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804709 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804712 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804714 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804717 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 19:00:19.088600 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804719 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804724 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804726 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804729 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804731 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804734 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804736 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804739 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804741 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804744 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804746 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804748 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804751 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804753 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804756 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804760 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804762 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804766 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804769 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 19:00:19.089132 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804772 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804774 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804777 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804779 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804782 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804784 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804787 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804789 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804792 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804794 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804797 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804800 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804802 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804805 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804809 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804811 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804814 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804816 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.804819 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804892 2571 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804899 2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 19:00:19.089599 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804905 2571 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804909 2571 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804913 2571 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804917 2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804921 2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804925 2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804928 2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804931 2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804934 2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804940 2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804943 2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804946 2571 flags.go:64] FLAG: --cgroup-root="" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804949 2571 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804953 2571 flags.go:64] FLAG: --client-ca-file="" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804957 2571 flags.go:64] FLAG: --cloud-provider="external" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804960 2571 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804964 2571 flags.go:64] FLAG: --cluster-domain="" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804967 2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804970 2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804973 2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804977 2571 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804980 2571 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804983 2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 19:00:19.090148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804986 2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804989 2571 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804994 2571 flags.go:64] FLAG: --contention-profiling="false" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.804997 2571 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805000 2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805003 2571 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805006 2571 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805010 2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805013 2571 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805016 2571 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805019 2571 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805022 2571 flags.go:64] FLAG: --enable-server="true" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805024 2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805028 2571 flags.go:64] FLAG: --event-burst="100" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805031 2571 flags.go:64] FLAG: --event-qps="50" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805034 2571 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805037 2571 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805040 2571 flags.go:64] FLAG: --eviction-hard="" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805045 2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805048 2571 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805051 2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805054 2571 flags.go:64] FLAG: --eviction-soft="" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805057 2571 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805060 2571 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805063 2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 19:00:19.090729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805066 2571 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805069 2571 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805072 2571 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805074 2571 flags.go:64] FLAG: --feature-gates="" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805078 2571 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805081 2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805084 2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805087 2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805090 2571 flags.go:64] FLAG: --healthz-port="10248" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805094 2571 flags.go:64] FLAG: --help="false" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805115 2571 flags.go:64] FLAG: --hostname-override="ip-10-0-131-229.ec2.internal" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805119 2571 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805122 2571 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805125 2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805129 2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805132 2571 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805135 2571 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805138 2571 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805140 2571 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805143 2571 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805146 2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805149 2571 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805152 2571 flags.go:64] FLAG: --kube-reserved="" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805155 2571 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 19:00:19.091395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805158 2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805162 2571 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805165 2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805168 2571 flags.go:64] FLAG: --lock-file="" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805171 2571 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805174 2571 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805177 2571 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805183 2571 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805186 2571 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805189 2571 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805191 2571 flags.go:64] FLAG: --logging-format="text" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805195 2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805198 2571 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805201 2571 flags.go:64] FLAG: --manifest-url="" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805204 2571 flags.go:64] FLAG: --manifest-url-header="" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805207 2571 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805210 2571 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805215 2571 flags.go:64] FLAG: --max-pods="110" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805218 2571 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805221 2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805224 2571 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805227 2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805230 2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805232 2571 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805235 2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 19:00:19.091984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805242 2571 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805245 2571 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805248 2571 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805251 2571 flags.go:64] FLAG: --pod-cidr="" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805254 2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805260 2571 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805263 2571 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805267 2571 flags.go:64] FLAG: --pods-per-core="0" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805269 2571 flags.go:64] FLAG: --port="10250" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805274 2571 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805277 2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-020781d05f17ba72b" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805280 2571 flags.go:64] FLAG: --qos-reserved="" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805283 2571 flags.go:64] FLAG: --read-only-port="10255" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805285 2571 flags.go:64] FLAG: --register-node="true" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805288 2571 flags.go:64] FLAG: --register-with-taints="" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805292 2571 flags.go:64] FLAG: --registry-burst="10" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805296 2571 flags.go:64] FLAG: --registry-qps="5" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805298 2571 flags.go:64] FLAG: --reserved-cpus="" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805301 2571 flags.go:64] FLAG: --reserved-memory="" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805305 2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805308 2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805311 2571 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805314 2571 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805316 2571 flags.go:64] FLAG: --runonce="false" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805321 2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 19:00:19.092612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805324 2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805327 2571 flags.go:64] FLAG: --seccomp-default="false" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805330 2571 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805333 2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805335 2571 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805338 2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805341 2571 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805344 2571 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805347 2571 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805350 2571 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805353 2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805356 2571 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805358 2571 flags.go:64] FLAG: --system-cgroups="" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805361 2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805366 2571 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805369 2571 flags.go:64] FLAG: --tls-cert-file="" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805373 2571 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805378 2571 flags.go:64] FLAG: --tls-min-version="" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805380 2571 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805383 2571 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805386 2571 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805388 2571 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805391 2571 flags.go:64] FLAG: --v="2" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805395 2571 flags.go:64] FLAG: --version="false" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805400 2571 flags.go:64] FLAG: --vmodule="" Jul 10 19:00:19.093231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805404 2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805407 2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805525 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805528 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805531 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805534 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805538 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805541 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805544 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805547 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805549 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805552 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805554 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805557 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805559 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805562 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805564 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805567 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805571 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805574 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 19:00:19.093870 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805577 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805580 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805583 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805587 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805590 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805592 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805596 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805599 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805602 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805605 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805608 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805611 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805615 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805618 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805620 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805623 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805626 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805628 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805632 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 19:00:19.094368 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805635 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805638 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805640 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805643 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805645 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805648 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805650 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805653 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805655 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805658 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805660 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805663 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805665 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805667 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805670 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805672 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805676 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805678 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805681 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805683 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 19:00:19.094822 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805686 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805689 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805691 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805694 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805696 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805699 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805702 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805705 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805707 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805710 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805714 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805717 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805721 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805724 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805727 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805729 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805733 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805736 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805739 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 19:00:19.095311 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805742 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805745 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805748 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805751 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805753 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805756 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805759 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805762 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805764 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805767 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805769 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805772 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805774 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805777 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805779 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805782 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805784 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805786 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805789 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805791 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 19:00:19.095764 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805794 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 19:00:19.096665 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805797 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 19:00:19.096665 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.805799 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 19:00:19.096665 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.805805 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 19:00:19.096665 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.812396 2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 19:00:19.096665 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.812414 2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 19:00:19.096665 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.813485 2571 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 19:00:19.096665 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.813492 2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 19:00:19.096665 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813561 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 19:00:19.096665 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813568 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 19:00:19.096665 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813572 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 19:00:19.096665 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813577 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 19:00:19.096665 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813580 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 19:00:19.096665 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813584 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 19:00:19.096665 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813588 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813591 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813593 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813596 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813599 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813603 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813607 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813611 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813615 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813618 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813621 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813625 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813628 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813632 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813635 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813638 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813641 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813643 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813646 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 19:00:19.097022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813650 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813654 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813657 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813660 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813664 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813670 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813675 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813678 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813681 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813684 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813687 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813690 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813692 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813695 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813699 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813703 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813705 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813708 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813711 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813714 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 19:00:19.097491 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813717 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813721 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813724 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813726 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813729 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813732 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813734 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813738 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813742 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813744 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813748 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813751 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813754 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813757 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813759 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813764 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813767 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813770 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813773 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813777 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813781 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 19:00:19.097995 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813783 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813786 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813788 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813791 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813795 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813799 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813801 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813806 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813811 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813814 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813816 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813819 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813822 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813824 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813827 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813830 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813834 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813837 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813840 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813843 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 19:00:19.098546 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813845 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 19:00:19.099186 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813855 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 19:00:19.099186 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813859 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 19:00:19.099186 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813862 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 19:00:19.099186 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813865 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 19:00:19.099186 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813867 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 19:00:19.099186 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813873 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 19:00:19.099186 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813876 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 19:00:19.099186 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813878 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 19:00:19.099186 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813882 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 19:00:19.099186 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813886 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 19:00:19.099186 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813889 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 19:00:19.099186 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.813892 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 19:00:19.099186 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.813898 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 19:00:19.099186 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814035 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814045 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814051 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814055 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814058 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814061 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814064 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814067 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814070 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814073 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814077 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814081 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814083 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814086 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814089 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814092 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814110 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814115 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814118 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 19:00:19.099566 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814120 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814123 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814126 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814130 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814133 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814136 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814140 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814142 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814145 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814149 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814152 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814154 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814157 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814161 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814164 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814166 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814169 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814171 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814174 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814176 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814179 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 19:00:19.100022 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814182 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814185 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814187 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814190 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814192 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814195 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814198 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814201 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814203 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814206 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814209 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814211 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814214 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814216 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814218 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814221 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814223 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814228 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814230 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 19:00:19.100553 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814233 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814235 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814238 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814241 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814244 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814247 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814250 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814252 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814255 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814257 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814261 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814264 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814267 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814270 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814272 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814275 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814277 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814280 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814283 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814285 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 19:00:19.100997 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814288 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814291 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814294 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814298 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814301 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814303 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814306 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814308 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814311 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814314 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814317 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814321 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814325 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814329 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814333 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814338 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814341 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814346 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814349 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 19:00:19.101554 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:18.814352 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 19:00:19.102017 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.814358 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 19:00:19.102017 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.814479 2571 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 19:00:19.102017 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.816675 2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 19:00:19.102017 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.818051 2571 server.go:1019] "Starting client certificate rotation" Jul 10 19:00:19.102017 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.818156 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 19:00:19.102017 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.818191 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 19:00:19.102017 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.820451 2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 19:00:19.102017 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.824469 2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 19:00:19.102017 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.835334 2571 log.go:25] "Validated CRI v1 runtime API" Jul 10 19:00:19.102017 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.840550 2571 log.go:25] "Validated CRI v1 image API" Jul 10 19:00:19.102017 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.841233 2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 19:00:19.102017 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.844288 2571 fs.go:135] Filesystem UUIDs: map[29968a3f-5e6e-457e-9a31-96d9bb039098:/dev/nvme0n1p3 4f0d8ad4-c29c-4228-ae83-b8c3414a95e7:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 10 19:00:19.102335 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.844308 2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 19:00:19.102370 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.851489 2571 manager.go:216] Machine: {Timestamp:2026-07-10 19:00:18.848387198 +0000 UTC m=+0.384729621 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100127 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2774abac3053fd9a51b02acc4c70a8 SystemUUID:ec2774ab-ac30-53fd-9a51-b02acc4c70a8 BootID:f1f1734f-7792-45b4-a3d2-58856af6e17a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ce:dc:f5:fa:9f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ce:dc:f5:fa:9f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:bf:a6:ad:80:64 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 19:00:19.102370 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.851635 2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 19:00:19.102370 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.851722 2571 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 19:00:19.102370 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.852613 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 19:00:19.102370 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.853904 2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 19:00:19.102370 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.853941 2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-229.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 19:00:19.102370 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.855043 2571 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.855053 2571 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.855060 2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.855721 2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.856472 2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.860012 2571 state_mem.go:36] "Initialized new in-memory state store" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.860318 2571 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.860348 2571 kubelet.go:486] "Attempting to sync node with API server" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.860357 2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.860369 2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.860378 2571 kubelet.go:398] "Adding apiserver pod source" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.860387 2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.860730 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.860746 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.863431 2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.864714 2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.864742 2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.865296 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.865310 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.865316 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.865321 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.865329 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.865335 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.865470 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.865476 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.865482 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.865488 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.865502 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.865510 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.865533 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.865538 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.868765 2571 server.go:1268] "Started kubelet" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.868941 2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.869042 2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.869130 2571 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.870122 2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.871560 2571 server.go:310] "Adding debug handlers to kubelet server" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:18.875005 2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.876474 2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.876490 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:18.876680 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.877112 2571 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.877127 2571 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.877196 2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.877510 2571 reconstruct.go:97] "Volume reconstruction finished" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.877522 2571 reconciler.go:29] "Reconciler: start to sync state" Jul 10 19:00:19.102573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.877805 2571 factory.go:55] Registering systemd factory Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.877817 2571 factory.go:223] Registration of the systemd container factory successfully Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.878282 2571 factory.go:153] Registering CRI-O factory Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.878296 2571 factory.go:223] Registration of the crio container factory successfully Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.878352 2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.878375 2571 factory.go:103] Registering Raw factory Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.878391 2571 manager.go:1196] Started watching for new ooms in manager Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.878906 2571 manager.go:349] Starting recovery of all containers Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:18.878707 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-229.ec2.internal.18c102cfc8f9fa66 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-229.ec2.internal,UID:ip-10-0-131-229.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-229.ec2.internal,},FirstTimestamp:2026-07-10 19:00:18.868738662 +0000 UTC m=+0.405081085,LastTimestamp:2026-07-10 19:00:18.868738662 +0000 UTC m=+0.405081085,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-229.ec2.internal,}" Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:18.883766 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:18.883827 2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-229.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:18.883877 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-229.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.884032 2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-229.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:18.884406 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.889412 2571 manager.go:354] Recovery completed Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.896010 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.897961 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientMemory" Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.898010 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.898023 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientPID" Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.898564 2571 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.898575 2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.898579 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.898590 2571 state_mem.go:36] "Initialized new in-memory state store" Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.899235 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientMemory" Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.899257 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 19:00:19.103717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.899266 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientPID" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:18.900197 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-229.ec2.internal.18c102cfcab85325 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-229.ec2.internal,UID:ip-10-0-131-229.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-229.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-229.ec2.internal,},FirstTimestamp:2026-07-10 19:00:18.897990437 +0000 UTC m=+0.434332868,LastTimestamp:2026-07-10 19:00:18.897990437 +0000 UTC m=+0.434332868,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-229.ec2.internal,}" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.900647 2571 policy_none.go:49] "None policy: Start" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.901113 2571 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.901133 2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.903120 2571 policy_none.go:47] "Start" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:18.911194 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-229.ec2.internal.18c102cfcab8bc49 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-229.ec2.internal,UID:ip-10-0-131-229.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-229.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-229.ec2.internal,},FirstTimestamp:2026-07-10 19:00:18.898017353 +0000 UTC m=+0.434359776,LastTimestamp:2026-07-10 19:00:18.898017353 +0000 UTC m=+0.434359776,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-229.ec2.internal,}" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.911264 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6pjsn" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:18.920688 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-229.ec2.internal.18c102cfcab8e3e2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-229.ec2.internal,UID:ip-10-0-131-229.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-229.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-229.ec2.internal,},FirstTimestamp:2026-07-10 19:00:18.89802749 +0000 UTC m=+0.434369913,LastTimestamp:2026-07-10 19:00:18.89802749 +0000 UTC m=+0.434369913,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-229.ec2.internal,}" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.921455 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6pjsn" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.939725 2571 manager.go:337] "Starting Device Plugin manager" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:18.939753 2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.939763 2571 server.go:87] "Starting device plugin registration server" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.939996 2571 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.940009 2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.940198 2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.940273 2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:18.940281 2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:18.940716 2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:18.940760 2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 19:00:19.104345 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.010036 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 19:00:19.104840 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.011488 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 19:00:19.104840 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.011520 2571 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 19:00:19.104840 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.011538 2571 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 19:00:19.104840 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:19.011566 2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 19:00:19.104840 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.013923 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 19:00:19.104840 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.040905 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 19:00:19.104840 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.041882 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientMemory" Jul 10 19:00:19.104840 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.041908 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 19:00:19.104840 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.041920 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientPID" Jul 10 19:00:19.104840 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.041940 2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-229.ec2.internal" Jul 10 19:00:19.104840 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.051116 2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-229.ec2.internal" Jul 10 19:00:19.104840 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:19.051134 2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-229.ec2.internal\": node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 19:00:19.104840 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:19.084574 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 19:00:19.112240 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.112222 2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal"] Jul 10 19:00:19.112309 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.112292 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 19:00:19.113087 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.113072 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientMemory" Jul 10 19:00:19.113181 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.113110 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 19:00:19.113181 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.113125 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientPID" Jul 10 19:00:19.115445 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.115428 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 19:00:19.115553 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.115538 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" Jul 10 19:00:19.115596 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.115573 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 19:00:19.116082 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.116068 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientMemory" Jul 10 19:00:19.116082 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.116079 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientMemory" Jul 10 19:00:19.116218 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.116113 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 19:00:19.116218 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.116120 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 19:00:19.116218 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.116129 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientPID" Jul 10 19:00:19.116218 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.116134 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientPID" Jul 10 19:00:19.118276 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.118261 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal" Jul 10 19:00:19.118355 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.118289 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 19:00:19.118890 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.118869 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientMemory" Jul 10 19:00:19.118890 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.118894 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 19:00:19.119047 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.118908 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientPID" Jul 10 19:00:19.159736 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:19.159720 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-229.ec2.internal\" not found" node="ip-10-0-131-229.ec2.internal" Jul 10 19:00:19.164031 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:19.164015 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-229.ec2.internal\" not found" node="ip-10-0-131-229.ec2.internal" Jul 10 19:00:19.183622 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.183598 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aa60a3a8ee1eea11634837cc945056ef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal\" (UID: \"aa60a3a8ee1eea11634837cc945056ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" Jul 10 19:00:19.183709 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.183629 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa60a3a8ee1eea11634837cc945056ef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal\" (UID: \"aa60a3a8ee1eea11634837cc945056ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" Jul 10 19:00:19.183709 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.183653 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fe6cda58f405aaf9348408762755bc73-config\") pod \"kube-apiserver-proxy-ip-10-0-131-229.ec2.internal\" (UID: \"fe6cda58f405aaf9348408762755bc73\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal" Jul 10 19:00:19.184768 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:19.184753 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 19:00:19.284712 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.284691 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aa60a3a8ee1eea11634837cc945056ef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal\" (UID: \"aa60a3a8ee1eea11634837cc945056ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" Jul 10 19:00:19.284791 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.284717 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa60a3a8ee1eea11634837cc945056ef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal\" (UID: \"aa60a3a8ee1eea11634837cc945056ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" Jul 10 19:00:19.284791 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.284740 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fe6cda58f405aaf9348408762755bc73-config\") pod \"kube-apiserver-proxy-ip-10-0-131-229.ec2.internal\" (UID: \"fe6cda58f405aaf9348408762755bc73\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal" Jul 10 19:00:19.284791 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.284786 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa60a3a8ee1eea11634837cc945056ef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal\" (UID: \"aa60a3a8ee1eea11634837cc945056ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" Jul 10 19:00:19.284877 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:19.284831 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 19:00:19.284877 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.284840 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aa60a3a8ee1eea11634837cc945056ef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal\" (UID: \"aa60a3a8ee1eea11634837cc945056ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" Jul 10 19:00:19.284932 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.284875 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fe6cda58f405aaf9348408762755bc73-config\") pod \"kube-apiserver-proxy-ip-10-0-131-229.ec2.internal\" (UID: \"fe6cda58f405aaf9348408762755bc73\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal" Jul 10 19:00:19.385374 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:19.385324 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 19:00:19.462532 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.462506 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" Jul 10 19:00:19.466627 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.466612 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal" Jul 10 19:00:19.485518 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:19.485497 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 19:00:19.585772 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:19.585749 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 19:00:19.686116 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:19.686042 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 19:00:19.786339 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:19.786312 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 19:00:19.817430 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.817411 2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 19:00:19.817976 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.817577 2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 19:00:19.875209 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.875192 2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 19:00:19.886428 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:19.886407 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 19:00:19.925919 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.925892 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 18:55:18 +0000 UTC" deadline="2028-04-12 04:35:42.747620752 +0000 UTC" Jul 10 19:00:19.925919 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.925918 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15393h35m22.82170597s" Jul 10 19:00:19.978213 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:19.978186 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe6cda58f405aaf9348408762755bc73.slice/crio-90e204c40d70696a2f27aca144d65ad378cfca9c2e11f2c40c450ae0b23bad22 WatchSource:0}: Error finding container 90e204c40d70696a2f27aca144d65ad378cfca9c2e11f2c40c450ae0b23bad22: Status 404 returned error can't find the container with id 90e204c40d70696a2f27aca144d65ad378cfca9c2e11f2c40c450ae0b23bad22 Jul 10 19:00:19.978464 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:19.978446 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa60a3a8ee1eea11634837cc945056ef.slice/crio-fc6d96c60ba7c195ed48d3e1c8c4da1bac5455205c5268dcbd6b0fdcb9267667 WatchSource:0}: Error finding container fc6d96c60ba7c195ed48d3e1c8c4da1bac5455205c5268dcbd6b0fdcb9267667: Status 404 returned error can't find the container with id fc6d96c60ba7c195ed48d3e1c8c4da1bac5455205c5268dcbd6b0fdcb9267667 Jul 10 19:00:19.982355 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:19.982342 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 19:00:19.986912 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:19.986897 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 19:00:20.014489 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.014448 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" event={"ID":"aa60a3a8ee1eea11634837cc945056ef","Type":"ContainerStarted","Data":"fc6d96c60ba7c195ed48d3e1c8c4da1bac5455205c5268dcbd6b0fdcb9267667"} Jul 10 19:00:20.015372 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.015349 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal" event={"ID":"fe6cda58f405aaf9348408762755bc73","Type":"ContainerStarted","Data":"90e204c40d70696a2f27aca144d65ad378cfca9c2e11f2c40c450ae0b23bad22"} Jul 10 19:00:20.051049 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.051033 2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 19:00:20.072136 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.072121 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 19:00:20.073653 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.073640 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientMemory" Jul 10 19:00:20.073735 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.073667 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 19:00:20.073735 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.073682 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientPID" Jul 10 19:00:20.087947 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:20.087926 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 19:00:20.188656 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:20.188632 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 19:00:20.193135 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.193123 2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 19:00:20.278064 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.278020 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" Jul 10 19:00:20.291801 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.291785 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 19:00:20.292807 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.292790 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal" Jul 10 19:00:20.302735 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.301472 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 19:00:20.691957 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.691877 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 19:00:20.861271 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.861238 2571 apiserver.go:52] "Watching apiserver" Jul 10 19:00:20.869771 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.869744 2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 19:00:20.871042 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.871008 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-kdk7z","openshift-image-registry/node-ca-hzqbp","openshift-network-diagnostics/network-check-target-vjklm","kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal","openshift-dns/node-resolver-vrqsx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal","openshift-multus/multus-additional-cni-plugins-vml84","openshift-multus/multus-dwjp4","openshift-multus/network-metrics-daemon-9p2rq","openshift-network-operator/iptables-alerter-wm79h","openshift-ovn-kubernetes/ovnkube-node-kgpdv","kube-system/global-pull-secret-syncer-4cdrm","kube-system/konnectivity-agent-xm2hk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n"] Jul 10 19:00:20.873576 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.873553 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.875759 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.875740 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hzqbp" Jul 10 19:00:20.876511 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.876367 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 19:00:20.876511 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.876415 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 19:00:20.876511 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.876476 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kf7ks\"" Jul 10 19:00:20.876824 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.876797 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 19:00:20.878979 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.878175 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wm79h" Jul 10 19:00:20.878979 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.878620 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gtkvf\"" Jul 10 19:00:20.878979 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.878681 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 19:00:20.878979 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.878755 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 19:00:20.878979 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.878883 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 19:00:20.880347 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.880328 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vrqsx" Jul 10 19:00:20.881460 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.881088 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-skhgz\"" Jul 10 19:00:20.881460 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.881227 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 19:00:20.881460 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.881288 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 19:00:20.881460 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.881372 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 19:00:20.882792 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.882541 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:20.882924 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.882900 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-p9chh\"" Jul 10 19:00:20.883175 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.883158 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 19:00:20.883259 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.883175 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 19:00:20.884671 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.884653 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dwjp4" Jul 10 19:00:20.884821 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.884800 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 19:00:20.885245 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.885227 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 19:00:20.885332 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.885310 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4688h\"" Jul 10 19:00:20.885433 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.885413 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 19:00:20.885673 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.885655 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 19:00:20.886199 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.886177 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 19:00:20.886934 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.886916 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 19:00:20.887351 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.887335 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zq5fg\"" Jul 10 19:00:20.887646 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.887626 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:20.887717 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:20.887696 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9p2rq" podUID="f911a124-c338-40f4-983f-c61a0d899d93" Jul 10 19:00:20.888433 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.888411 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 19:00:20.889864 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.889845 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:20.889955 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:20.889918 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjklm" podUID="182da83f-63cb-4013-9868-4e95e297c58a" Jul 10 19:00:20.893008 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.892810 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.893617 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.893594 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/965c2d5d-ff87-46d2-9de2-7062e810316f-host\") pod \"node-ca-hzqbp\" (UID: \"965c2d5d-ff87-46d2-9de2-7062e810316f\") " pod="openshift-image-registry/node-ca-hzqbp" Jul 10 19:00:20.893718 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.893634 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f4s6\" (UniqueName: \"kubernetes.io/projected/965c2d5d-ff87-46d2-9de2-7062e810316f-kube-api-access-2f4s6\") pod \"node-ca-hzqbp\" (UID: \"965c2d5d-ff87-46d2-9de2-7062e810316f\") " pod="openshift-image-registry/node-ca-hzqbp" Jul 10 19:00:20.893718 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.893669 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/18f374ad-202d-43e8-b083-aabf526ff3d7-hosts-file\") pod \"node-resolver-vrqsx\" (UID: \"18f374ad-202d-43e8-b083-aabf526ff3d7\") " pod="openshift-dns/node-resolver-vrqsx" Jul 10 19:00:20.893718 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.893700 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-tuned\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.893877 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.893728 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-kubernetes\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.893877 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.893781 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-sysctl-d\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.893877 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.893803 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-sysctl-conf\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.893877 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.893833 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47153e6e-af80-45ea-977c-dca2a2a8eb4c-host-slash\") pod \"iptables-alerter-wm79h\" (UID: \"47153e6e-af80-45ea-977c-dca2a2a8eb4c\") " pod="openshift-network-operator/iptables-alerter-wm79h" Jul 10 19:00:20.893877 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.893861 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp7gz\" (UniqueName: \"kubernetes.io/projected/47153e6e-af80-45ea-977c-dca2a2a8eb4c-kube-api-access-sp7gz\") pod \"iptables-alerter-wm79h\" (UID: \"47153e6e-af80-45ea-977c-dca2a2a8eb4c\") " pod="openshift-network-operator/iptables-alerter-wm79h" Jul 10 19:00:20.894209 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.893892 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18f374ad-202d-43e8-b083-aabf526ff3d7-tmp-dir\") pod \"node-resolver-vrqsx\" (UID: \"18f374ad-202d-43e8-b083-aabf526ff3d7\") " pod="openshift-dns/node-resolver-vrqsx" Jul 10 19:00:20.894209 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.893919 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-lib-modules\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.894209 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.893942 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/47153e6e-af80-45ea-977c-dca2a2a8eb4c-iptables-alerter-script\") pod \"iptables-alerter-wm79h\" (UID: \"47153e6e-af80-45ea-977c-dca2a2a8eb4c\") " pod="openshift-network-operator/iptables-alerter-wm79h" Jul 10 19:00:20.894209 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.893970 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-systemd\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.894209 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.893997 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-run\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.894459 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.894271 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-sys\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.894930 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.894775 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:20.894930 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:20.894858 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4cdrm" podUID="a7f690e5-5f82-4573-bc6e-a7ed083f6fbc" Jul 10 19:00:20.895615 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.895503 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-var-lib-kubelet\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.895615 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.895544 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-host\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.895615 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.895568 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd6aa889-75be-41f0-84f1-b480208c62ea-tmp\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.895811 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.895614 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/965c2d5d-ff87-46d2-9de2-7062e810316f-serviceca\") pod \"node-ca-hzqbp\" (UID: \"965c2d5d-ff87-46d2-9de2-7062e810316f\") " pod="openshift-image-registry/node-ca-hzqbp" Jul 10 19:00:20.895811 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.895655 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxc7t\" (UniqueName: \"kubernetes.io/projected/18f374ad-202d-43e8-b083-aabf526ff3d7-kube-api-access-wxc7t\") pod \"node-resolver-vrqsx\" (UID: \"18f374ad-202d-43e8-b083-aabf526ff3d7\") " pod="openshift-dns/node-resolver-vrqsx" Jul 10 19:00:20.895811 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.895681 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-modprobe-d\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.895811 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.895720 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-sysconfig\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.895811 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.895760 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6647c\" (UniqueName: \"kubernetes.io/projected/dd6aa889-75be-41f0-84f1-b480208c62ea-kube-api-access-6647c\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.896434 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.896369 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 19:00:20.896849 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.896828 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nbsjf\"" Jul 10 19:00:20.896932 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.896898 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 19:00:20.897038 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.897023 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 19:00:20.897133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.897094 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 19:00:20.897293 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.897278 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 19:00:20.897859 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.897839 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xm2hk" Jul 10 19:00:20.898004 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.897987 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 19:00:20.900547 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.900507 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 19:00:20.900889 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.900871 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-q6jds\"" Jul 10 19:00:20.901152 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.901132 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:20.901329 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.901312 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 19:00:20.904161 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.903959 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 19:00:20.904271 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.904251 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rf7wg\"" Jul 10 19:00:20.904271 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.904264 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 19:00:20.904366 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.904264 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 19:00:20.914669 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.914648 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-94pfj" Jul 10 19:00:20.924085 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.924045 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-94pfj" Jul 10 19:00:20.979050 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.979025 2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 19:00:20.996178 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996151 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9fb6edd9-1cae-495c-9a2a-2da429edff2c-agent-certs\") pod \"konnectivity-agent-xm2hk\" (UID: \"9fb6edd9-1cae-495c-9a2a-2da429edff2c\") " pod="kube-system/konnectivity-agent-xm2hk" Jul 10 19:00:20.996317 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996185 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-cni-netd\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.996317 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996215 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/965c2d5d-ff87-46d2-9de2-7062e810316f-serviceca\") pod \"node-ca-hzqbp\" (UID: \"965c2d5d-ff87-46d2-9de2-7062e810316f\") " pod="openshift-image-registry/node-ca-hzqbp" Jul 10 19:00:20.996317 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996264 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47153e6e-af80-45ea-977c-dca2a2a8eb4c-host-slash\") pod \"iptables-alerter-wm79h\" (UID: \"47153e6e-af80-45ea-977c-dca2a2a8eb4c\") " pod="openshift-network-operator/iptables-alerter-wm79h" Jul 10 19:00:20.996317 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996294 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-sys-fs\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:20.996524 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996320 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c52c2896-855d-422d-82db-504e4d38fec6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:20.996524 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996344 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c52c2896-855d-422d-82db-504e4d38fec6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:20.996524 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996361 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47153e6e-af80-45ea-977c-dca2a2a8eb4c-host-slash\") pod \"iptables-alerter-wm79h\" (UID: \"47153e6e-af80-45ea-977c-dca2a2a8eb4c\") " pod="openshift-network-operator/iptables-alerter-wm79h" Jul 10 19:00:20.996524 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996366 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-host-run-netns\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:20.996524 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996406 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-systemd-units\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.996524 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996429 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-multus-cni-dir\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:20.996524 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996462 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs\") pod \"network-metrics-daemon-9p2rq\" (UID: \"f911a124-c338-40f4-983f-c61a0d899d93\") " pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:20.996524 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996483 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-run-openvswitch\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.996524 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996509 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-node-log\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.996983 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996532 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f217ac86-726b-4051-a7b4-aea1fe005f49-ovn-node-metrics-cert\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.996983 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996552 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-sysconfig\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.996983 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996572 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6647c\" (UniqueName: \"kubernetes.io/projected/dd6aa889-75be-41f0-84f1-b480208c62ea-kube-api-access-6647c\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.996983 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996597 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f4s6\" (UniqueName: \"kubernetes.io/projected/965c2d5d-ff87-46d2-9de2-7062e810316f-kube-api-access-2f4s6\") pod \"node-ca-hzqbp\" (UID: \"965c2d5d-ff87-46d2-9de2-7062e810316f\") " pod="openshift-image-registry/node-ca-hzqbp" Jul 10 19:00:20.996983 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996644 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/965c2d5d-ff87-46d2-9de2-7062e810316f-serviceca\") pod \"node-ca-hzqbp\" (UID: \"965c2d5d-ff87-46d2-9de2-7062e810316f\") " pod="openshift-image-registry/node-ca-hzqbp" Jul 10 19:00:20.996983 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.996697 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-sysconfig\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.997327 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997071 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/822fc44e-edb1-4d03-a5ce-32ac417aebac-cni-binary-copy\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:20.997327 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997095 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-multus-socket-dir-parent\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:20.997327 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997137 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c52c2896-855d-422d-82db-504e4d38fec6-system-cni-dir\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:20.997327 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997153 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c52c2896-855d-422d-82db-504e4d38fec6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:20.997327 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997171 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd6aa889-75be-41f0-84f1-b480208c62ea-tmp\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.997327 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997264 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-lib-modules\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.997617 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997296 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjqgm\" (UniqueName: \"kubernetes.io/projected/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-kube-api-access-bjqgm\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:20.997617 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997380 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-multus-conf-dir\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:20.997617 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997395 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-kubelet-config\") pod \"global-pull-secret-syncer-4cdrm\" (UID: \"a7f690e5-5f82-4573-bc6e-a7ed083f6fbc\") " pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:20.997617 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997424 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-etc-openvswitch\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.997617 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997444 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-lib-modules\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.997617 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997447 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-systemd\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.997617 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997479 2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 19:00:20.997617 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997511 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-systemd\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.997617 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997530 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-var-lib-kubelet\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.997617 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997482 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-var-lib-kubelet\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.997617 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997567 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c52c2896-855d-422d-82db-504e4d38fec6-cnibin\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:20.997617 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997587 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c52c2896-855d-422d-82db-504e4d38fec6-cni-binary-copy\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:20.997617 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997613 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8qhq\" (UniqueName: \"kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq\") pod \"network-check-target-vjklm\" (UID: \"182da83f-63cb-4013-9868-4e95e297c58a\") " pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:20.998226 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997636 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-run-systemd\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.998226 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997676 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-run-ovn-kubernetes\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.998226 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997699 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-cni-bin\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.998226 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997724 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxc7t\" (UniqueName: \"kubernetes.io/projected/18f374ad-202d-43e8-b083-aabf526ff3d7-kube-api-access-wxc7t\") pod \"node-resolver-vrqsx\" (UID: \"18f374ad-202d-43e8-b083-aabf526ff3d7\") " pod="openshift-dns/node-resolver-vrqsx" Jul 10 19:00:20.998226 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997783 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-modprobe-d\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.998226 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997863 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-device-dir\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:20.998226 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997886 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-modprobe-d\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.998226 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997891 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-cnibin\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:20.998226 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997924 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-host-run-k8s-cni-cncf-io\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:20.998226 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997953 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/18f374ad-202d-43e8-b083-aabf526ff3d7-hosts-file\") pod \"node-resolver-vrqsx\" (UID: \"18f374ad-202d-43e8-b083-aabf526ff3d7\") " pod="openshift-dns/node-resolver-vrqsx" Jul 10 19:00:20.998226 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.997996 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-host-run-multus-certs\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:20.998226 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998024 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret\") pod \"global-pull-secret-syncer-4cdrm\" (UID: \"a7f690e5-5f82-4573-bc6e-a7ed083f6fbc\") " pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:20.998226 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998078 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-kubelet\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.998226 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998092 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/18f374ad-202d-43e8-b083-aabf526ff3d7-hosts-file\") pod \"node-resolver-vrqsx\" (UID: \"18f374ad-202d-43e8-b083-aabf526ff3d7\") " pod="openshift-dns/node-resolver-vrqsx" Jul 10 19:00:20.998955 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998134 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-log-socket\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.998955 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998184 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-kubernetes\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.998955 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998219 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp7gz\" (UniqueName: \"kubernetes.io/projected/47153e6e-af80-45ea-977c-dca2a2a8eb4c-kube-api-access-sp7gz\") pod \"iptables-alerter-wm79h\" (UID: \"47153e6e-af80-45ea-977c-dca2a2a8eb4c\") " pod="openshift-network-operator/iptables-alerter-wm79h" Jul 10 19:00:20.998955 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998257 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-kubernetes\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.998955 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998290 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18f374ad-202d-43e8-b083-aabf526ff3d7-tmp-dir\") pod \"node-resolver-vrqsx\" (UID: \"18f374ad-202d-43e8-b083-aabf526ff3d7\") " pod="openshift-dns/node-resolver-vrqsx" Jul 10 19:00:20.998955 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998317 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmstl\" (UniqueName: \"kubernetes.io/projected/822fc44e-edb1-4d03-a5ce-32ac417aebac-kube-api-access-kmstl\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:20.998955 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998340 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wbpx\" (UniqueName: \"kubernetes.io/projected/f911a124-c338-40f4-983f-c61a0d899d93-kube-api-access-8wbpx\") pod \"network-metrics-daemon-9p2rq\" (UID: \"f911a124-c338-40f4-983f-c61a0d899d93\") " pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:20.998955 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998361 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f217ac86-726b-4051-a7b4-aea1fe005f49-ovnkube-config\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.998955 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998421 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f217ac86-726b-4051-a7b4-aea1fe005f49-env-overrides\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.998955 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998458 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-run\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.998955 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998481 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-etc-selinux\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:20.998955 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998505 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-system-cni-dir\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:20.998955 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998535 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvrtl\" (UniqueName: \"kubernetes.io/projected/f217ac86-726b-4051-a7b4-aea1fe005f49-kube-api-access-vvrtl\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.998955 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998562 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/965c2d5d-ff87-46d2-9de2-7062e810316f-host\") pod \"node-ca-hzqbp\" (UID: \"965c2d5d-ff87-46d2-9de2-7062e810316f\") " pod="openshift-image-registry/node-ca-hzqbp" Jul 10 19:00:20.999652 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998572 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-run\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.999652 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998572 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18f374ad-202d-43e8-b083-aabf526ff3d7-tmp-dir\") pod \"node-resolver-vrqsx\" (UID: \"18f374ad-202d-43e8-b083-aabf526ff3d7\") " pod="openshift-dns/node-resolver-vrqsx" Jul 10 19:00:20.999652 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998601 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-registration-dir\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:20.999652 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998608 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/965c2d5d-ff87-46d2-9de2-7062e810316f-host\") pod \"node-ca-hzqbp\" (UID: \"965c2d5d-ff87-46d2-9de2-7062e810316f\") " pod="openshift-image-registry/node-ca-hzqbp" Jul 10 19:00:20.999652 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998630 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-host-var-lib-cni-multus\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:20.999652 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998654 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-slash\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.999652 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998677 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-run-netns\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.999652 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998700 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f217ac86-726b-4051-a7b4-aea1fe005f49-ovnkube-script-lib\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.999652 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998723 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:20.999652 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998746 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-host-var-lib-kubelet\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:20.999652 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998768 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-var-lib-openvswitch\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:20.999652 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998794 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-sysctl-d\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.999652 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998816 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-sysctl-conf\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:20.999652 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998832 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-etc-kubernetes\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.000380 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998849 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.000380 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998873 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/47153e6e-af80-45ea-977c-dca2a2a8eb4c-iptables-alerter-script\") pod \"iptables-alerter-wm79h\" (UID: \"47153e6e-af80-45ea-977c-dca2a2a8eb4c\") " pod="openshift-network-operator/iptables-alerter-wm79h" Jul 10 19:00:21.000380 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998887 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-sys\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:21.000380 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998905 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-host\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:21.000380 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998925 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-sysctl-d\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:21.000380 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.998955 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-host\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:21.000380 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.999001 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-sys\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:21.000380 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.999025 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-sysctl-conf\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:21.000380 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.999056 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c52c2896-855d-422d-82db-504e4d38fec6-os-release\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.000380 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.999083 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrpdd\" (UniqueName: \"kubernetes.io/projected/c52c2896-855d-422d-82db-504e4d38fec6-kube-api-access-zrpdd\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.000380 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.999143 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-hostroot\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.000380 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.999162 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-dbus\") pod \"global-pull-secret-syncer-4cdrm\" (UID: \"a7f690e5-5f82-4573-bc6e-a7ed083f6fbc\") " pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:21.000380 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.999180 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-run-ovn\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.000380 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.999205 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-socket-dir\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:21.001043 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.999224 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-os-release\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.001043 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.999243 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-host-var-lib-cni-bin\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.001043 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.999264 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/822fc44e-edb1-4d03-a5ce-32ac417aebac-multus-daemon-config\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.001043 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.999285 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9fb6edd9-1cae-495c-9a2a-2da429edff2c-konnectivity-ca\") pod \"konnectivity-agent-xm2hk\" (UID: \"9fb6edd9-1cae-495c-9a2a-2da429edff2c\") " pod="kube-system/konnectivity-agent-xm2hk" Jul 10 19:00:21.001043 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.999310 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-tuned\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:21.001043 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:20.999458 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/47153e6e-af80-45ea-977c-dca2a2a8eb4c-iptables-alerter-script\") pod \"iptables-alerter-wm79h\" (UID: \"47153e6e-af80-45ea-977c-dca2a2a8eb4c\") " pod="openshift-network-operator/iptables-alerter-wm79h" Jul 10 19:00:21.002364 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.002331 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dd6aa889-75be-41f0-84f1-b480208c62ea-etc-tuned\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:21.002605 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.002584 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd6aa889-75be-41f0-84f1-b480208c62ea-tmp\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:21.008440 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.008359 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f4s6\" (UniqueName: \"kubernetes.io/projected/965c2d5d-ff87-46d2-9de2-7062e810316f-kube-api-access-2f4s6\") pod \"node-ca-hzqbp\" (UID: \"965c2d5d-ff87-46d2-9de2-7062e810316f\") " pod="openshift-image-registry/node-ca-hzqbp" Jul 10 19:00:21.009095 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.008652 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6647c\" (UniqueName: \"kubernetes.io/projected/dd6aa889-75be-41f0-84f1-b480208c62ea-kube-api-access-6647c\") pod \"tuned-kdk7z\" (UID: \"dd6aa889-75be-41f0-84f1-b480208c62ea\") " pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:21.013773 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.013488 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxc7t\" (UniqueName: \"kubernetes.io/projected/18f374ad-202d-43e8-b083-aabf526ff3d7-kube-api-access-wxc7t\") pod \"node-resolver-vrqsx\" (UID: \"18f374ad-202d-43e8-b083-aabf526ff3d7\") " pod="openshift-dns/node-resolver-vrqsx" Jul 10 19:00:21.013773 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.013635 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp7gz\" (UniqueName: \"kubernetes.io/projected/47153e6e-af80-45ea-977c-dca2a2a8eb4c-kube-api-access-sp7gz\") pod \"iptables-alerter-wm79h\" (UID: \"47153e6e-af80-45ea-977c-dca2a2a8eb4c\") " pod="openshift-network-operator/iptables-alerter-wm79h" Jul 10 19:00:21.099834 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.099802 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-etc-openvswitch\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.100006 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.099846 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c52c2896-855d-422d-82db-504e4d38fec6-cnibin\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.100006 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.099868 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c52c2896-855d-422d-82db-504e4d38fec6-cni-binary-copy\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.100006 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.099894 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8qhq\" (UniqueName: \"kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq\") pod \"network-check-target-vjklm\" (UID: \"182da83f-63cb-4013-9868-4e95e297c58a\") " pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:21.100006 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.099917 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-run-systemd\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.100006 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.099923 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c52c2896-855d-422d-82db-504e4d38fec6-cnibin\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.100006 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.099939 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-run-ovn-kubernetes\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.100006 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.099924 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-etc-openvswitch\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.100006 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.099984 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-run-ovn-kubernetes\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.100006 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.099986 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-run-systemd\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.100006 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100010 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-cni-bin\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.100474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100056 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-device-dir\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:21.100474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100066 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-cni-bin\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.100474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100080 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-cnibin\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.100474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100123 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-host-run-k8s-cni-cncf-io\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.100474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100147 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-host-run-multus-certs\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.100474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100156 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-cnibin\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.100474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100154 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-device-dir\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:21.100474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100178 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret\") pod \"global-pull-secret-syncer-4cdrm\" (UID: \"a7f690e5-5f82-4573-bc6e-a7ed083f6fbc\") " pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:21.100474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100194 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-host-run-k8s-cni-cncf-io\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.100474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100204 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-kubelet\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.100474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100208 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-host-run-multus-certs\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.100474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100238 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-log-socket\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.100474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100254 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-kubelet\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.100474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100266 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmstl\" (UniqueName: \"kubernetes.io/projected/822fc44e-edb1-4d03-a5ce-32ac417aebac-kube-api-access-kmstl\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.100474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100291 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wbpx\" (UniqueName: \"kubernetes.io/projected/f911a124-c338-40f4-983f-c61a0d899d93-kube-api-access-8wbpx\") pod \"network-metrics-daemon-9p2rq\" (UID: \"f911a124-c338-40f4-983f-c61a0d899d93\") " pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:21.101172 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100295 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-log-socket\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.101172 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:21.100292 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 19:00:21.101172 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100314 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f217ac86-726b-4051-a7b4-aea1fe005f49-ovnkube-config\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.101172 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:21.100386 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret podName:a7f690e5-5f82-4573-bc6e-a7ed083f6fbc nodeName:}" failed. No retries permitted until 2026-07-10 19:00:21.600346258 +0000 UTC m=+3.136688668 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret") pod "global-pull-secret-syncer-4cdrm" (UID: "a7f690e5-5f82-4573-bc6e-a7ed083f6fbc") : object "kube-system"/"original-pull-secret" not registered Jul 10 19:00:21.101172 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100515 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f217ac86-726b-4051-a7b4-aea1fe005f49-env-overrides\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.101172 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100532 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c52c2896-855d-422d-82db-504e4d38fec6-cni-binary-copy\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.101172 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100560 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-etc-selinux\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:21.101172 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100586 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-system-cni-dir\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.101172 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100642 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-system-cni-dir\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.101172 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100723 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvrtl\" (UniqueName: \"kubernetes.io/projected/f217ac86-726b-4051-a7b4-aea1fe005f49-kube-api-access-vvrtl\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.101172 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100776 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-registration-dir\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:21.101172 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100785 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-etc-selinux\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:21.101172 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100801 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-host-var-lib-cni-multus\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.101172 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100826 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-slash\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.101768 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100838 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f217ac86-726b-4051-a7b4-aea1fe005f49-ovnkube-config\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.101768 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100856 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-run-netns\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.101768 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100881 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f217ac86-726b-4051-a7b4-aea1fe005f49-ovnkube-script-lib\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.101768 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100892 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-host-var-lib-cni-multus\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.101768 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100904 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:21.101768 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100916 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f217ac86-726b-4051-a7b4-aea1fe005f49-env-overrides\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.101768 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100929 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-host-var-lib-kubelet\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.101768 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100939 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-registration-dir\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:21.101768 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100974 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-var-lib-openvswitch\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.101768 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.100983 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-host-var-lib-kubelet\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.101768 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101004 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:21.101768 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101004 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-etc-kubernetes\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.101768 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101019 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-run-netns\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.101768 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101045 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-slash\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.102433 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101047 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.102433 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101058 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-var-lib-openvswitch\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.102433 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101080 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c52c2896-855d-422d-82db-504e4d38fec6-os-release\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.102433 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101084 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.102433 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101122 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrpdd\" (UniqueName: \"kubernetes.io/projected/c52c2896-855d-422d-82db-504e4d38fec6-kube-api-access-zrpdd\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.102433 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101136 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-etc-kubernetes\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.102433 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101155 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-hostroot\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.102433 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101162 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c52c2896-855d-422d-82db-504e4d38fec6-os-release\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.102433 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101179 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-dbus\") pod \"global-pull-secret-syncer-4cdrm\" (UID: \"a7f690e5-5f82-4573-bc6e-a7ed083f6fbc\") " pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:21.102433 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101203 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-run-ovn\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.102433 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101216 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-hostroot\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.102433 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101236 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-socket-dir\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:21.102433 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101261 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-os-release\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.102433 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101266 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-run-ovn\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.103133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101283 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-host-var-lib-cni-bin\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.103133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101307 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/822fc44e-edb1-4d03-a5ce-32ac417aebac-multus-daemon-config\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.103133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101307 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-dbus\") pod \"global-pull-secret-syncer-4cdrm\" (UID: \"a7f690e5-5f82-4573-bc6e-a7ed083f6fbc\") " pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:21.103133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101338 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9fb6edd9-1cae-495c-9a2a-2da429edff2c-konnectivity-ca\") pod \"konnectivity-agent-xm2hk\" (UID: \"9fb6edd9-1cae-495c-9a2a-2da429edff2c\") " pod="kube-system/konnectivity-agent-xm2hk" Jul 10 19:00:21.103133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101358 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-socket-dir\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:21.103133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101363 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9fb6edd9-1cae-495c-9a2a-2da429edff2c-agent-certs\") pod \"konnectivity-agent-xm2hk\" (UID: \"9fb6edd9-1cae-495c-9a2a-2da429edff2c\") " pod="kube-system/konnectivity-agent-xm2hk" Jul 10 19:00:21.103133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101371 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-os-release\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.103133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101391 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-cni-netd\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.103133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101414 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-host-var-lib-cni-bin\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.103133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101419 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-sys-fs\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:21.103133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101421 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f217ac86-726b-4051-a7b4-aea1fe005f49-ovnkube-script-lib\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.103133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101465 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-sys-fs\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:21.103133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101463 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-host-cni-netd\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.103133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101541 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c52c2896-855d-422d-82db-504e4d38fec6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.103788 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101574 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c52c2896-855d-422d-82db-504e4d38fec6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.103788 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101598 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-host-run-netns\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.103788 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101631 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-systemd-units\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.103788 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101653 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-multus-cni-dir\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.103788 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101676 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs\") pod \"network-metrics-daemon-9p2rq\" (UID: \"f911a124-c338-40f4-983f-c61a0d899d93\") " pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:21.103788 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101684 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-host-run-netns\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.103788 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101697 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-run-openvswitch\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.103788 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101737 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-run-openvswitch\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.103788 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101747 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-multus-cni-dir\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.103788 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101747 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c52c2896-855d-422d-82db-504e4d38fec6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.103788 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101783 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-systemd-units\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.103788 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101785 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-node-log\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.103788 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:21.101815 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:00:21.103788 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101838 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f217ac86-726b-4051-a7b4-aea1fe005f49-ovn-node-metrics-cert\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.104389 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:21.101861 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs podName:f911a124-c338-40f4-983f-c61a0d899d93 nodeName:}" failed. No retries permitted until 2026-07-10 19:00:21.601846418 +0000 UTC m=+3.138188840 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs") pod "network-metrics-daemon-9p2rq" (UID: "f911a124-c338-40f4-983f-c61a0d899d93") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:00:21.104389 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101878 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9fb6edd9-1cae-495c-9a2a-2da429edff2c-konnectivity-ca\") pod \"konnectivity-agent-xm2hk\" (UID: \"9fb6edd9-1cae-495c-9a2a-2da429edff2c\") " pod="kube-system/konnectivity-agent-xm2hk" Jul 10 19:00:21.104389 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101896 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f217ac86-726b-4051-a7b4-aea1fe005f49-node-log\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.104389 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101892 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/822fc44e-edb1-4d03-a5ce-32ac417aebac-cni-binary-copy\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.104389 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101948 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-multus-socket-dir-parent\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.104389 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.101987 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c52c2896-855d-422d-82db-504e4d38fec6-system-cni-dir\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.104389 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.102001 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/822fc44e-edb1-4d03-a5ce-32ac417aebac-multus-daemon-config\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.104389 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.102022 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-multus-socket-dir-parent\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.104389 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.102031 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c52c2896-855d-422d-82db-504e4d38fec6-system-cni-dir\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.104389 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.102061 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c52c2896-855d-422d-82db-504e4d38fec6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.104389 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.102120 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c52c2896-855d-422d-82db-504e4d38fec6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.104389 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.102154 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjqgm\" (UniqueName: \"kubernetes.io/projected/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-kube-api-access-bjqgm\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:21.104389 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.102179 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-multus-conf-dir\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.104796 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.102203 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-kubelet-config\") pod \"global-pull-secret-syncer-4cdrm\" (UID: \"a7f690e5-5f82-4573-bc6e-a7ed083f6fbc\") " pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:21.104796 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.102274 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-kubelet-config\") pod \"global-pull-secret-syncer-4cdrm\" (UID: \"a7f690e5-5f82-4573-bc6e-a7ed083f6fbc\") " pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:21.104796 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.102276 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/822fc44e-edb1-4d03-a5ce-32ac417aebac-multus-conf-dir\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.104796 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.102409 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/822fc44e-edb1-4d03-a5ce-32ac417aebac-cni-binary-copy\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.104796 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.102624 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c52c2896-855d-422d-82db-504e4d38fec6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.104796 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.104177 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f217ac86-726b-4051-a7b4-aea1fe005f49-ovn-node-metrics-cert\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.104796 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.104466 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9fb6edd9-1cae-495c-9a2a-2da429edff2c-agent-certs\") pod \"konnectivity-agent-xm2hk\" (UID: \"9fb6edd9-1cae-495c-9a2a-2da429edff2c\") " pod="kube-system/konnectivity-agent-xm2hk" Jul 10 19:00:21.108361 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:21.108201 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 19:00:21.108361 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:21.108223 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 19:00:21.108361 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:21.108236 2571 projected.go:196] Error preparing data for projected volume kube-api-access-b8qhq for pod openshift-network-diagnostics/network-check-target-vjklm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:00:21.108361 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:21.108289 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq podName:182da83f-63cb-4013-9868-4e95e297c58a nodeName:}" failed. No retries permitted until 2026-07-10 19:00:21.608273379 +0000 UTC m=+3.144615808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b8qhq" (UniqueName: "kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq") pod "network-check-target-vjklm" (UID: "182da83f-63cb-4013-9868-4e95e297c58a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:00:21.110008 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.109964 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvrtl\" (UniqueName: \"kubernetes.io/projected/f217ac86-726b-4051-a7b4-aea1fe005f49-kube-api-access-vvrtl\") pod \"ovnkube-node-kgpdv\" (UID: \"f217ac86-726b-4051-a7b4-aea1fe005f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.110296 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.110276 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmstl\" (UniqueName: \"kubernetes.io/projected/822fc44e-edb1-4d03-a5ce-32ac417aebac-kube-api-access-kmstl\") pod \"multus-dwjp4\" (UID: \"822fc44e-edb1-4d03-a5ce-32ac417aebac\") " pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.110395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.110371 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjqgm\" (UniqueName: \"kubernetes.io/projected/5cf9833f-d47d-41ea-b998-67c5d7b38ab9-kube-api-access-bjqgm\") pod \"aws-ebs-csi-driver-node-f5h2n\" (UID: \"5cf9833f-d47d-41ea-b998-67c5d7b38ab9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:21.110685 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.110666 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrpdd\" (UniqueName: \"kubernetes.io/projected/c52c2896-855d-422d-82db-504e4d38fec6-kube-api-access-zrpdd\") pod \"multus-additional-cni-plugins-vml84\" (UID: \"c52c2896-855d-422d-82db-504e4d38fec6\") " pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.111545 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.111522 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wbpx\" (UniqueName: \"kubernetes.io/projected/f911a124-c338-40f4-983f-c61a0d899d93-kube-api-access-8wbpx\") pod \"network-metrics-daemon-9p2rq\" (UID: \"f911a124-c338-40f4-983f-c61a0d899d93\") " pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:21.187144 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.187113 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" Jul 10 19:00:21.196701 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.196677 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hzqbp" Jul 10 19:00:21.208329 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.208308 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wm79h" Jul 10 19:00:21.215010 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.214970 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vrqsx" Jul 10 19:00:21.221549 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.221528 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vml84" Jul 10 19:00:21.230040 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.230022 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dwjp4" Jul 10 19:00:21.236652 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.236636 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:21.244211 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.244187 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xm2hk" Jul 10 19:00:21.250697 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.250682 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" Jul 10 19:00:21.606079 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.605791 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs\") pod \"network-metrics-daemon-9p2rq\" (UID: \"f911a124-c338-40f4-983f-c61a0d899d93\") " pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:21.606257 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.606085 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret\") pod \"global-pull-secret-syncer-4cdrm\" (UID: \"a7f690e5-5f82-4573-bc6e-a7ed083f6fbc\") " pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:21.606257 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:21.605943 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:00:21.606257 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:21.606188 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs podName:f911a124-c338-40f4-983f-c61a0d899d93 nodeName:}" failed. No retries permitted until 2026-07-10 19:00:22.606166247 +0000 UTC m=+4.142508658 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs") pod "network-metrics-daemon-9p2rq" (UID: "f911a124-c338-40f4-983f-c61a0d899d93") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:00:21.606257 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:21.606213 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 19:00:21.606467 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:21.606265 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret podName:a7f690e5-5f82-4573-bc6e-a7ed083f6fbc nodeName:}" failed. No retries permitted until 2026-07-10 19:00:22.606248939 +0000 UTC m=+4.142591362 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret") pod "global-pull-secret-syncer-4cdrm" (UID: "a7f690e5-5f82-4573-bc6e-a7ed083f6fbc") : object "kube-system"/"original-pull-secret" not registered Jul 10 19:00:21.706588 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.706554 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8qhq\" (UniqueName: \"kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq\") pod \"network-check-target-vjklm\" (UID: \"182da83f-63cb-4013-9868-4e95e297c58a\") " pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:21.706725 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:21.706707 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 19:00:21.706778 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:21.706727 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 19:00:21.706778 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:21.706736 2571 projected.go:196] Error preparing data for projected volume kube-api-access-b8qhq for pod openshift-network-diagnostics/network-check-target-vjklm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:00:21.706885 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:21.706792 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq podName:182da83f-63cb-4013-9868-4e95e297c58a nodeName:}" failed. No retries permitted until 2026-07-10 19:00:22.706774795 +0000 UTC m=+4.243117226 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b8qhq" (UniqueName: "kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq") pod "network-check-target-vjklm" (UID: "182da83f-63cb-4013-9868-4e95e297c58a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:00:21.927202 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.927077 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 18:55:20 +0000 UTC" deadline="2028-03-16 10:33:06.919542858 +0000 UTC" Jul 10 19:00:21.927202 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:21.927147 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14751h32m44.992401584s" Jul 10 19:00:22.012757 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:22.012734 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:22.012880 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:22.012827 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4cdrm" podUID="a7f690e5-5f82-4573-bc6e-a7ed083f6fbc" Jul 10 19:00:22.096236 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:22.096038 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd6aa889_75be_41f0_84f1_b480208c62ea.slice/crio-b880a4cdca0a6266e063529de1d9c69e1755f2dd12f387abff6c4e25a3e6cc30 WatchSource:0}: Error finding container b880a4cdca0a6266e063529de1d9c69e1755f2dd12f387abff6c4e25a3e6cc30: Status 404 returned error can't find the container with id b880a4cdca0a6266e063529de1d9c69e1755f2dd12f387abff6c4e25a3e6cc30 Jul 10 19:00:22.096738 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:22.096626 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cf9833f_d47d_41ea_b998_67c5d7b38ab9.slice/crio-56cf0507c0a4ea8ebb356f46ecfdff764ae0ea08f91a8b209f37edb4507261c5 WatchSource:0}: Error finding container 56cf0507c0a4ea8ebb356f46ecfdff764ae0ea08f91a8b209f37edb4507261c5: Status 404 returned error can't find the container with id 56cf0507c0a4ea8ebb356f46ecfdff764ae0ea08f91a8b209f37edb4507261c5 Jul 10 19:00:22.099488 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:22.099456 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc52c2896_855d_422d_82db_504e4d38fec6.slice/crio-4e00d7d30d13f47bf25ddd487e43d0a1b14c9820df23ad0f2db2bc1dccc0a1f5 WatchSource:0}: Error finding container 4e00d7d30d13f47bf25ddd487e43d0a1b14c9820df23ad0f2db2bc1dccc0a1f5: Status 404 returned error can't find the container with id 4e00d7d30d13f47bf25ddd487e43d0a1b14c9820df23ad0f2db2bc1dccc0a1f5 Jul 10 19:00:22.102229 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:22.102208 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18f374ad_202d_43e8_b083_aabf526ff3d7.slice/crio-12947cefe3ee7b60c00b594bb15e6d3cd7516ccfa6881305d53c498f19e47678 WatchSource:0}: Error finding container 12947cefe3ee7b60c00b594bb15e6d3cd7516ccfa6881305d53c498f19e47678: Status 404 returned error can't find the container with id 12947cefe3ee7b60c00b594bb15e6d3cd7516ccfa6881305d53c498f19e47678 Jul 10 19:00:22.102625 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:22.102603 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf217ac86_726b_4051_a7b4_aea1fe005f49.slice/crio-e74d563817b9e70f1ac4ead1889a798d35c0c03e3b1c4f5a8fb2b8b6be7d7355 WatchSource:0}: Error finding container e74d563817b9e70f1ac4ead1889a798d35c0c03e3b1c4f5a8fb2b8b6be7d7355: Status 404 returned error can't find the container with id e74d563817b9e70f1ac4ead1889a798d35c0c03e3b1c4f5a8fb2b8b6be7d7355 Jul 10 19:00:22.104356 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:22.104151 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod965c2d5d_ff87_46d2_9de2_7062e810316f.slice/crio-18d524f8646820046c91b1f01add4abefbce9df6ddad78b5425ef12ac6edf0e1 WatchSource:0}: Error finding container 18d524f8646820046c91b1f01add4abefbce9df6ddad78b5425ef12ac6edf0e1: Status 404 returned error can't find the container with id 18d524f8646820046c91b1f01add4abefbce9df6ddad78b5425ef12ac6edf0e1 Jul 10 19:00:22.105450 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:22.105423 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fb6edd9_1cae_495c_9a2a_2da429edff2c.slice/crio-eb4ffec7e8e6a2391cd589fa3b246345b516c0ba520b9b9dd57ad2fea04adf02 WatchSource:0}: Error finding container eb4ffec7e8e6a2391cd589fa3b246345b516c0ba520b9b9dd57ad2fea04adf02: Status 404 returned error can't find the container with id eb4ffec7e8e6a2391cd589fa3b246345b516c0ba520b9b9dd57ad2fea04adf02 Jul 10 19:00:22.108247 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:22.108228 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47153e6e_af80_45ea_977c_dca2a2a8eb4c.slice/crio-ae7719cd96ac9828c68474c7f923359420d91c1f90b7a5bf3f0d3cc5ce3edf2a WatchSource:0}: Error finding container ae7719cd96ac9828c68474c7f923359420d91c1f90b7a5bf3f0d3cc5ce3edf2a: Status 404 returned error can't find the container with id ae7719cd96ac9828c68474c7f923359420d91c1f90b7a5bf3f0d3cc5ce3edf2a Jul 10 19:00:22.612091 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:22.611899 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs\") pod \"network-metrics-daemon-9p2rq\" (UID: \"f911a124-c338-40f4-983f-c61a0d899d93\") " pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:22.612091 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:22.611972 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret\") pod \"global-pull-secret-syncer-4cdrm\" (UID: \"a7f690e5-5f82-4573-bc6e-a7ed083f6fbc\") " pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:22.612286 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:22.612087 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:00:22.612286 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:22.612182 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs podName:f911a124-c338-40f4-983f-c61a0d899d93 nodeName:}" failed. No retries permitted until 2026-07-10 19:00:24.612160882 +0000 UTC m=+6.148503294 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs") pod "network-metrics-daemon-9p2rq" (UID: "f911a124-c338-40f4-983f-c61a0d899d93") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:00:22.612286 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:22.612094 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 19:00:22.612286 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:22.612266 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret podName:a7f690e5-5f82-4573-bc6e-a7ed083f6fbc nodeName:}" failed. No retries permitted until 2026-07-10 19:00:24.612247819 +0000 UTC m=+6.148590238 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret") pod "global-pull-secret-syncer-4cdrm" (UID: "a7f690e5-5f82-4573-bc6e-a7ed083f6fbc") : object "kube-system"/"original-pull-secret" not registered Jul 10 19:00:22.713485 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:22.712757 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8qhq\" (UniqueName: \"kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq\") pod \"network-check-target-vjklm\" (UID: \"182da83f-63cb-4013-9868-4e95e297c58a\") " pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:22.713485 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:22.712988 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 19:00:22.713485 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:22.713012 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 19:00:22.713485 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:22.713024 2571 projected.go:196] Error preparing data for projected volume kube-api-access-b8qhq for pod openshift-network-diagnostics/network-check-target-vjklm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:00:22.713485 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:22.713083 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq podName:182da83f-63cb-4013-9868-4e95e297c58a nodeName:}" failed. No retries permitted until 2026-07-10 19:00:24.713065326 +0000 UTC m=+6.249407738 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b8qhq" (UniqueName: "kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq") pod "network-check-target-vjklm" (UID: "182da83f-63cb-4013-9868-4e95e297c58a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:00:22.932177 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:22.928250 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 18:55:20 +0000 UTC" deadline="2028-04-08 20:25:38.943716708 +0000 UTC" Jul 10 19:00:22.932177 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:22.928289 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15313h25m16.015432089s" Jul 10 19:00:23.014655 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:23.014629 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:23.014812 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:23.014759 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjklm" podUID="182da83f-63cb-4013-9868-4e95e297c58a" Jul 10 19:00:23.015178 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:23.015154 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:23.015278 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:23.015260 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9p2rq" podUID="f911a124-c338-40f4-983f-c61a0d899d93" Jul 10 19:00:23.026031 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:23.025998 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" event={"ID":"dd6aa889-75be-41f0-84f1-b480208c62ea","Type":"ContainerStarted","Data":"b880a4cdca0a6266e063529de1d9c69e1755f2dd12f387abff6c4e25a3e6cc30"} Jul 10 19:00:23.038197 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:23.038128 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xm2hk" event={"ID":"9fb6edd9-1cae-495c-9a2a-2da429edff2c","Type":"ContainerStarted","Data":"eb4ffec7e8e6a2391cd589fa3b246345b516c0ba520b9b9dd57ad2fea04adf02"} Jul 10 19:00:23.053824 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:23.053789 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hzqbp" event={"ID":"965c2d5d-ff87-46d2-9de2-7062e810316f","Type":"ContainerStarted","Data":"18d524f8646820046c91b1f01add4abefbce9df6ddad78b5425ef12ac6edf0e1"} Jul 10 19:00:23.056351 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:23.056256 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" event={"ID":"f217ac86-726b-4051-a7b4-aea1fe005f49","Type":"ContainerStarted","Data":"e74d563817b9e70f1ac4ead1889a798d35c0c03e3b1c4f5a8fb2b8b6be7d7355"} Jul 10 19:00:23.073429 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:23.073347 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vml84" event={"ID":"c52c2896-855d-422d-82db-504e4d38fec6","Type":"ContainerStarted","Data":"4e00d7d30d13f47bf25ddd487e43d0a1b14c9820df23ad0f2db2bc1dccc0a1f5"} Jul 10 19:00:23.081008 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:23.080090 2571 generic.go:356] "Generic (PLEG): container finished" podID="aa60a3a8ee1eea11634837cc945056ef" containerID="54a036dcb28a5bbfc9d8c30485f3a0f49820f5c1c74e6a773e8be2eb670ebdff" exitCode=0 Jul 10 19:00:23.081008 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:23.080196 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" event={"ID":"aa60a3a8ee1eea11634837cc945056ef","Type":"ContainerDied","Data":"54a036dcb28a5bbfc9d8c30485f3a0f49820f5c1c74e6a773e8be2eb670ebdff"} Jul 10 19:00:23.087036 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:23.086619 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal" event={"ID":"fe6cda58f405aaf9348408762755bc73","Type":"ContainerStarted","Data":"ebec48124fea12d1f056b4b65b8d8387a54f4e9033f455c0e4680dd81b9c534a"} Jul 10 19:00:23.097983 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:23.097951 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wm79h" event={"ID":"47153e6e-af80-45ea-977c-dca2a2a8eb4c","Type":"ContainerStarted","Data":"ae7719cd96ac9828c68474c7f923359420d91c1f90b7a5bf3f0d3cc5ce3edf2a"} Jul 10 19:00:23.106799 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:23.106772 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dwjp4" event={"ID":"822fc44e-edb1-4d03-a5ce-32ac417aebac","Type":"ContainerStarted","Data":"f51b98a76befc0be93c22a0998094ce65d05819ac87e3deab3b6ef6f96acc6c0"} Jul 10 19:00:23.118787 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:23.117894 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal" podStartSLOduration=3.117878455 podStartE2EDuration="3.117878455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:00:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:00:23.117595321 +0000 UTC m=+4.653937758" watchObservedRunningTime="2026-07-10 19:00:23.117878455 +0000 UTC m=+4.654220888" Jul 10 19:00:23.119893 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:23.119828 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vrqsx" event={"ID":"18f374ad-202d-43e8-b083-aabf526ff3d7","Type":"ContainerStarted","Data":"12947cefe3ee7b60c00b594bb15e6d3cd7516ccfa6881305d53c498f19e47678"} Jul 10 19:00:23.125507 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:23.125436 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" event={"ID":"5cf9833f-d47d-41ea-b998-67c5d7b38ab9","Type":"ContainerStarted","Data":"56cf0507c0a4ea8ebb356f46ecfdff764ae0ea08f91a8b209f37edb4507261c5"} Jul 10 19:00:24.012832 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:24.012727 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:24.013301 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:24.012851 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4cdrm" podUID="a7f690e5-5f82-4573-bc6e-a7ed083f6fbc" Jul 10 19:00:24.152149 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:24.151918 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" event={"ID":"aa60a3a8ee1eea11634837cc945056ef","Type":"ContainerStarted","Data":"74b9324b311fc7526dad1298dd5f7533a88ee963448c935ed855989631ed92bb"} Jul 10 19:00:24.190952 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:24.190896 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" podStartSLOduration=4.190876986 podStartE2EDuration="4.190876986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:00:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:00:24.190640367 +0000 UTC m=+5.726982800" watchObservedRunningTime="2026-07-10 19:00:24.190876986 +0000 UTC m=+5.727219420" Jul 10 19:00:24.628356 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:24.628320 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs\") pod \"network-metrics-daemon-9p2rq\" (UID: \"f911a124-c338-40f4-983f-c61a0d899d93\") " pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:24.628538 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:24.628397 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret\") pod \"global-pull-secret-syncer-4cdrm\" (UID: \"a7f690e5-5f82-4573-bc6e-a7ed083f6fbc\") " pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:24.628598 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:24.628547 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 19:00:24.628651 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:24.628621 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret podName:a7f690e5-5f82-4573-bc6e-a7ed083f6fbc nodeName:}" failed. No retries permitted until 2026-07-10 19:00:28.628592723 +0000 UTC m=+10.164935136 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret") pod "global-pull-secret-syncer-4cdrm" (UID: "a7f690e5-5f82-4573-bc6e-a7ed083f6fbc") : object "kube-system"/"original-pull-secret" not registered Jul 10 19:00:24.629084 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:24.629007 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:00:24.629084 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:24.629059 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs podName:f911a124-c338-40f4-983f-c61a0d899d93 nodeName:}" failed. No retries permitted until 2026-07-10 19:00:28.629044142 +0000 UTC m=+10.165386559 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs") pod "network-metrics-daemon-9p2rq" (UID: "f911a124-c338-40f4-983f-c61a0d899d93") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:00:24.729759 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:24.729024 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8qhq\" (UniqueName: \"kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq\") pod \"network-check-target-vjklm\" (UID: \"182da83f-63cb-4013-9868-4e95e297c58a\") " pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:24.729759 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:24.729249 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 19:00:24.729759 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:24.729274 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 19:00:24.729759 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:24.729287 2571 projected.go:196] Error preparing data for projected volume kube-api-access-b8qhq for pod openshift-network-diagnostics/network-check-target-vjklm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:00:24.729759 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:24.729389 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq podName:182da83f-63cb-4013-9868-4e95e297c58a nodeName:}" failed. No retries permitted until 2026-07-10 19:00:28.729324589 +0000 UTC m=+10.265667014 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b8qhq" (UniqueName: "kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq") pod "network-check-target-vjklm" (UID: "182da83f-63cb-4013-9868-4e95e297c58a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:00:25.017501 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:25.017431 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:25.017923 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:25.017563 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjklm" podUID="182da83f-63cb-4013-9868-4e95e297c58a" Jul 10 19:00:25.017990 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:25.017930 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:25.018067 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:25.018037 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9p2rq" podUID="f911a124-c338-40f4-983f-c61a0d899d93" Jul 10 19:00:26.012688 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:26.012228 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:26.012688 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:26.012352 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4cdrm" podUID="a7f690e5-5f82-4573-bc6e-a7ed083f6fbc" Jul 10 19:00:27.012242 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:27.012211 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:27.012704 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:27.012364 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9p2rq" podUID="f911a124-c338-40f4-983f-c61a0d899d93" Jul 10 19:00:27.012704 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:27.012438 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:27.012704 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:27.012551 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjklm" podUID="182da83f-63cb-4013-9868-4e95e297c58a" Jul 10 19:00:28.012802 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:28.012762 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:28.013256 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:28.012903 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4cdrm" podUID="a7f690e5-5f82-4573-bc6e-a7ed083f6fbc" Jul 10 19:00:28.660709 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:28.660673 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret\") pod \"global-pull-secret-syncer-4cdrm\" (UID: \"a7f690e5-5f82-4573-bc6e-a7ed083f6fbc\") " pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:28.660879 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:28.660743 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs\") pod \"network-metrics-daemon-9p2rq\" (UID: \"f911a124-c338-40f4-983f-c61a0d899d93\") " pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:28.660879 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:28.660864 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:00:28.660988 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:28.660919 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs podName:f911a124-c338-40f4-983f-c61a0d899d93 nodeName:}" failed. No retries permitted until 2026-07-10 19:00:36.660901824 +0000 UTC m=+18.197244237 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs") pod "network-metrics-daemon-9p2rq" (UID: "f911a124-c338-40f4-983f-c61a0d899d93") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:00:28.661054 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:28.660982 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 19:00:28.661121 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:28.661052 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret podName:a7f690e5-5f82-4573-bc6e-a7ed083f6fbc nodeName:}" failed. No retries permitted until 2026-07-10 19:00:36.661034442 +0000 UTC m=+18.197376854 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret") pod "global-pull-secret-syncer-4cdrm" (UID: "a7f690e5-5f82-4573-bc6e-a7ed083f6fbc") : object "kube-system"/"original-pull-secret" not registered Jul 10 19:00:28.761209 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:28.761176 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8qhq\" (UniqueName: \"kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq\") pod \"network-check-target-vjklm\" (UID: \"182da83f-63cb-4013-9868-4e95e297c58a\") " pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:28.761401 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:28.761379 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 19:00:28.761468 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:28.761411 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 19:00:28.761468 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:28.761427 2571 projected.go:196] Error preparing data for projected volume kube-api-access-b8qhq for pod openshift-network-diagnostics/network-check-target-vjklm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:00:28.761581 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:28.761488 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq podName:182da83f-63cb-4013-9868-4e95e297c58a nodeName:}" failed. No retries permitted until 2026-07-10 19:00:36.761467567 +0000 UTC m=+18.297809991 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b8qhq" (UniqueName: "kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq") pod "network-check-target-vjklm" (UID: "182da83f-63cb-4013-9868-4e95e297c58a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:00:29.013251 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:29.013173 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:29.013693 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:29.013294 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9p2rq" podUID="f911a124-c338-40f4-983f-c61a0d899d93" Jul 10 19:00:29.013693 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:29.013363 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:29.013693 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:29.013496 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjklm" podUID="182da83f-63cb-4013-9868-4e95e297c58a" Jul 10 19:00:30.012907 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:30.012535 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:30.012907 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:30.012666 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4cdrm" podUID="a7f690e5-5f82-4573-bc6e-a7ed083f6fbc" Jul 10 19:00:31.012108 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:31.012066 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:31.012571 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:31.012226 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9p2rq" podUID="f911a124-c338-40f4-983f-c61a0d899d93" Jul 10 19:00:31.012571 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:31.012564 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:31.012679 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:31.012653 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjklm" podUID="182da83f-63cb-4013-9868-4e95e297c58a" Jul 10 19:00:32.011910 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:32.011883 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:32.012082 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:32.011986 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4cdrm" podUID="a7f690e5-5f82-4573-bc6e-a7ed083f6fbc" Jul 10 19:00:33.012531 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:33.012482 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:33.012982 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:33.012531 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:33.012982 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:33.012632 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9p2rq" podUID="f911a124-c338-40f4-983f-c61a0d899d93" Jul 10 19:00:33.012982 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:33.012757 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjklm" podUID="182da83f-63cb-4013-9868-4e95e297c58a" Jul 10 19:00:34.012042 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:34.012009 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:34.012215 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:34.012151 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4cdrm" podUID="a7f690e5-5f82-4573-bc6e-a7ed083f6fbc" Jul 10 19:00:35.012679 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:35.012647 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:35.013156 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:35.012772 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjklm" podUID="182da83f-63cb-4013-9868-4e95e297c58a" Jul 10 19:00:35.013156 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:35.012828 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:35.013156 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:35.012947 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9p2rq" podUID="f911a124-c338-40f4-983f-c61a0d899d93" Jul 10 19:00:36.012801 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:36.012765 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:36.013290 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:36.012897 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4cdrm" podUID="a7f690e5-5f82-4573-bc6e-a7ed083f6fbc" Jul 10 19:00:36.730063 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:36.729978 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs\") pod \"network-metrics-daemon-9p2rq\" (UID: \"f911a124-c338-40f4-983f-c61a0d899d93\") " pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:36.730063 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:36.730036 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret\") pod \"global-pull-secret-syncer-4cdrm\" (UID: \"a7f690e5-5f82-4573-bc6e-a7ed083f6fbc\") " pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:36.730268 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:36.730142 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 19:00:36.730268 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:36.730203 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret podName:a7f690e5-5f82-4573-bc6e-a7ed083f6fbc nodeName:}" failed. No retries permitted until 2026-07-10 19:00:52.730185768 +0000 UTC m=+34.266528181 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret") pod "global-pull-secret-syncer-4cdrm" (UID: "a7f690e5-5f82-4573-bc6e-a7ed083f6fbc") : object "kube-system"/"original-pull-secret" not registered Jul 10 19:00:36.730268 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:36.730142 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:00:36.730452 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:36.730310 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs podName:f911a124-c338-40f4-983f-c61a0d899d93 nodeName:}" failed. No retries permitted until 2026-07-10 19:00:52.730290178 +0000 UTC m=+34.266632604 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs") pod "network-metrics-daemon-9p2rq" (UID: "f911a124-c338-40f4-983f-c61a0d899d93") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:00:36.830811 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:36.830774 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8qhq\" (UniqueName: \"kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq\") pod \"network-check-target-vjklm\" (UID: \"182da83f-63cb-4013-9868-4e95e297c58a\") " pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:36.830989 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:36.830926 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 19:00:36.830989 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:36.830944 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 19:00:36.830989 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:36.830956 2571 projected.go:196] Error preparing data for projected volume kube-api-access-b8qhq for pod openshift-network-diagnostics/network-check-target-vjklm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:00:36.831126 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:36.831007 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq podName:182da83f-63cb-4013-9868-4e95e297c58a nodeName:}" failed. No retries permitted until 2026-07-10 19:00:52.830992263 +0000 UTC m=+34.367334696 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b8qhq" (UniqueName: "kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq") pod "network-check-target-vjklm" (UID: "182da83f-63cb-4013-9868-4e95e297c58a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:00:37.012909 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:37.012824 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:37.012909 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:37.012871 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:37.013379 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:37.012956 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjklm" podUID="182da83f-63cb-4013-9868-4e95e297c58a" Jul 10 19:00:37.013379 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:37.013133 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9p2rq" podUID="f911a124-c338-40f4-983f-c61a0d899d93" Jul 10 19:00:38.011714 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:38.011681 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:38.011877 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:38.011793 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4cdrm" podUID="a7f690e5-5f82-4573-bc6e-a7ed083f6fbc" Jul 10 19:00:39.013586 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:39.013381 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:39.014014 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:39.013449 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:39.014014 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:39.013664 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjklm" podUID="182da83f-63cb-4013-9868-4e95e297c58a" Jul 10 19:00:39.014014 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:39.013783 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9p2rq" podUID="f911a124-c338-40f4-983f-c61a0d899d93" Jul 10 19:00:40.012206 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:40.012172 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:40.012387 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:40.012293 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4cdrm" podUID="a7f690e5-5f82-4573-bc6e-a7ed083f6fbc" Jul 10 19:00:41.012076 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.012053 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:41.012770 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.012060 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:41.012770 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:41.012181 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjklm" podUID="182da83f-63cb-4013-9868-4e95e297c58a" Jul 10 19:00:41.012770 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:41.012232 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9p2rq" podUID="f911a124-c338-40f4-983f-c61a0d899d93" Jul 10 19:00:41.187724 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.187698 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xm2hk" event={"ID":"9fb6edd9-1cae-495c-9a2a-2da429edff2c","Type":"ContainerStarted","Data":"f23eb901aca5aba3c36278caa776499f3016f0dd44ca7b90c69632ce85c7bf30"} Jul 10 19:00:41.189152 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.189127 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hzqbp" event={"ID":"965c2d5d-ff87-46d2-9de2-7062e810316f","Type":"ContainerStarted","Data":"d89a3205d8f563e04011ee3b672b7add1c75a27f72cbe91f29bb7ba6839d9e4b"} Jul 10 19:00:41.191063 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.191046 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:00:41.191429 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.191375 2571 generic.go:356] "Generic (PLEG): container finished" podID="f217ac86-726b-4051-a7b4-aea1fe005f49" containerID="53124ca94593f723572c89abf234ce368e9520d07dfad51201258a599e896148" exitCode=1 Jul 10 19:00:41.191498 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.191438 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" event={"ID":"f217ac86-726b-4051-a7b4-aea1fe005f49","Type":"ContainerStarted","Data":"8013b63ddd41a91a502096ec7dec6aadfc79cc6001d79d1967ed0076337a68ea"} Jul 10 19:00:41.191554 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.191496 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" event={"ID":"f217ac86-726b-4051-a7b4-aea1fe005f49","Type":"ContainerStarted","Data":"11502610baef47514951b516e6bdab336cbe17e50966df5baeaf9ca8b82472c7"} Jul 10 19:00:41.191554 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.191545 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" event={"ID":"f217ac86-726b-4051-a7b4-aea1fe005f49","Type":"ContainerDied","Data":"53124ca94593f723572c89abf234ce368e9520d07dfad51201258a599e896148"} Jul 10 19:00:41.191626 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.191596 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" event={"ID":"f217ac86-726b-4051-a7b4-aea1fe005f49","Type":"ContainerStarted","Data":"ac8c69c217824f54134c874af7aba11f9fab136aca597fbc5995142af00ac907"} Jul 10 19:00:41.192964 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.192808 2571 generic.go:356] "Generic (PLEG): container finished" podID="c52c2896-855d-422d-82db-504e4d38fec6" containerID="1d3b2b84e59980983472d90684189ba55a93ec2c99ffdd548e16e08e32fe1527" exitCode=0 Jul 10 19:00:41.193035 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.192891 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vml84" event={"ID":"c52c2896-855d-422d-82db-504e4d38fec6","Type":"ContainerDied","Data":"1d3b2b84e59980983472d90684189ba55a93ec2c99ffdd548e16e08e32fe1527"} Jul 10 19:00:41.194867 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.194729 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dwjp4" event={"ID":"822fc44e-edb1-4d03-a5ce-32ac417aebac","Type":"ContainerStarted","Data":"2039bfdbfae094fd4319fa6c35c0343fda976c24f69d6511e57633e3deb5ad10"} Jul 10 19:00:41.199205 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.199172 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vrqsx" event={"ID":"18f374ad-202d-43e8-b083-aabf526ff3d7","Type":"ContainerStarted","Data":"d1fdc6365a98f8fb0c180df5b93ffa1ad48a61ba5cc699fb5720f7fbbdaf9fec"} Jul 10 19:00:41.200549 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.200519 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" event={"ID":"5cf9833f-d47d-41ea-b998-67c5d7b38ab9","Type":"ContainerStarted","Data":"7f60a4cfe98f457ec2d01b37f0b640c335916de0dce9f01d995dc5b95d6063a6"} Jul 10 19:00:41.202218 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.202198 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" event={"ID":"dd6aa889-75be-41f0-84f1-b480208c62ea","Type":"ContainerStarted","Data":"117fccf9ff97224809f61b03fc2062206b8ad24d74a4d6557433d727415f4169"} Jul 10 19:00:41.209596 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.209560 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xm2hk" podStartSLOduration=8.009645595 podStartE2EDuration="22.209541812s" totalImagesPullingTime="14.199896217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:00:41.209340232 +0000 UTC m=+22.745682665" watchObservedRunningTime="2026-07-10 19:00:41.209541812 +0000 UTC m=+22.745884243" Jul 10 19:00:41.247331 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.247291 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kdk7z" podStartSLOduration=4.659082961 podStartE2EDuration="23.247277184s" totalImagesPullingTime="18.588194223s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:00:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:00:41.230966627 +0000 UTC m=+22.767309059" watchObservedRunningTime="2026-07-10 19:00:41.247277184 +0000 UTC m=+22.783619616" Jul 10 19:00:41.247430 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.247378 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hzqbp" podStartSLOduration=4.69219336 podStartE2EDuration="23.247374338s" totalImagesPullingTime="18.555180978s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:00:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:00:41.246854768 +0000 UTC m=+22.783197200" watchObservedRunningTime="2026-07-10 19:00:41.247374338 +0000 UTC m=+22.783716770" Jul 10 19:00:41.264933 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.264897 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vrqsx" podStartSLOduration=3.706396137 podStartE2EDuration="22.264882945s" totalImagesPullingTime="18.558486808s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:00:41.264398033 +0000 UTC m=+22.800740464" watchObservedRunningTime="2026-07-10 19:00:41.264882945 +0000 UTC m=+22.801225378" Jul 10 19:00:41.313142 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:41.313083 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dwjp4" podStartSLOduration=3.683503113 podStartE2EDuration="22.313065071s" totalImagesPullingTime="18.629561958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:00:41.312810448 +0000 UTC m=+22.849152880" watchObservedRunningTime="2026-07-10 19:00:41.313065071 +0000 UTC m=+22.849407505" Jul 10 19:00:42.012771 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:42.012708 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:42.013219 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:42.012845 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4cdrm" podUID="a7f690e5-5f82-4573-bc6e-a7ed083f6fbc" Jul 10 19:00:42.208371 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:42.208341 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:00:42.208792 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:42.208759 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" event={"ID":"f217ac86-726b-4051-a7b4-aea1fe005f49","Type":"ContainerStarted","Data":"ce6ada0973e4c8818f762832e39f7c8cd6fa2bec8d1e7e9a27ef9e7b978ca2c4"} Jul 10 19:00:42.208913 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:42.208833 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" event={"ID":"f217ac86-726b-4051-a7b4-aea1fe005f49","Type":"ContainerStarted","Data":"0d70b31ff99dfec7c97e101b607cc0f250c64aaa04803bedf6461b8cf9d336b4"} Jul 10 19:00:42.210295 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:42.210243 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wm79h" event={"ID":"47153e6e-af80-45ea-977c-dca2a2a8eb4c","Type":"ContainerStarted","Data":"caee4175094c0db30704b50ff4a88a0daf60080ed25a8006e35df12abdc2e1da"} Jul 10 19:00:42.228464 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:42.228423 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wm79h" podStartSLOduration=4.655578946 podStartE2EDuration="23.228409192s" totalImagesPullingTime="18.572830246s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:00:42.228162926 +0000 UTC m=+23.764505359" watchObservedRunningTime="2026-07-10 19:00:42.228409192 +0000 UTC m=+23.764751620" Jul 10 19:00:42.574062 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:42.573875 2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 19:00:42.952675 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:42.952540 2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T19:00:42.574060461Z","UUID":"0a0cdb28-0cda-4154-ad1b-00e5ccc52519","Handler":null,"Name":"","Endpoint":""} Jul 10 19:00:42.954014 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:42.953995 2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 19:00:42.954331 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:42.954031 2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 19:00:43.012517 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:43.012484 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:43.012685 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:43.012484 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:43.012835 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:43.012798 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9p2rq" podUID="f911a124-c338-40f4-983f-c61a0d899d93" Jul 10 19:00:43.013223 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:43.012930 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjklm" podUID="182da83f-63cb-4013-9868-4e95e297c58a" Jul 10 19:00:43.214573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:43.214487 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" event={"ID":"5cf9833f-d47d-41ea-b998-67c5d7b38ab9","Type":"ContainerStarted","Data":"f66318da5a694bf1ad978f8f12834a871e088742535dc9bda83abd0ca00f60f8"} Jul 10 19:00:44.011862 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:44.011840 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:44.011976 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:44.011937 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4cdrm" podUID="a7f690e5-5f82-4573-bc6e-a7ed083f6fbc" Jul 10 19:00:44.219643 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:44.219562 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" event={"ID":"5cf9833f-d47d-41ea-b998-67c5d7b38ab9","Type":"ContainerStarted","Data":"21657ca11f2e354ed410109a0a2de0ce51328be7b8292b127541a7249d18b28d"} Jul 10 19:00:44.223229 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:44.223211 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:00:44.223598 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:44.223572 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" event={"ID":"f217ac86-726b-4051-a7b4-aea1fe005f49","Type":"ContainerStarted","Data":"7b42a8ee1dc82fe0f66ac93cee62d64eacffc8a7c038de5a89856b876cdc66db"} Jul 10 19:00:44.238881 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:44.238840 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f5h2n" podStartSLOduration=3.711109644 podStartE2EDuration="25.238828804s" totalImagesPullingTime="21.52771916s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:00:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:00:44.23832923 +0000 UTC m=+25.774671653" watchObservedRunningTime="2026-07-10 19:00:44.238828804 +0000 UTC m=+25.775171236" Jul 10 19:00:45.012568 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:45.012539 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:45.012743 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:45.012542 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:45.012743 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:45.012671 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9p2rq" podUID="f911a124-c338-40f4-983f-c61a0d899d93" Jul 10 19:00:45.012743 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:45.012715 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjklm" podUID="182da83f-63cb-4013-9868-4e95e297c58a" Jul 10 19:00:45.342227 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:45.342188 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xm2hk" Jul 10 19:00:45.737910 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:45.737749 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xm2hk" Jul 10 19:00:45.738506 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:45.738490 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xm2hk" Jul 10 19:00:46.012417 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:46.012331 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:46.012540 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:46.012425 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4cdrm" podUID="a7f690e5-5f82-4573-bc6e-a7ed083f6fbc" Jul 10 19:00:46.231991 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:46.231968 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:00:46.232351 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:46.232320 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" event={"ID":"f217ac86-726b-4051-a7b4-aea1fe005f49","Type":"ContainerStarted","Data":"1375894eedf9087124a52d9c77794ab60a2b651dee2fd7a121d02bb48348366d"} Jul 10 19:00:46.232656 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:46.232626 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:46.232817 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:46.232773 2571 scope.go:117] "RemoveContainer" containerID="53124ca94593f723572c89abf234ce368e9520d07dfad51201258a599e896148" Jul 10 19:00:46.234432 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:46.234406 2571 generic.go:356] "Generic (PLEG): container finished" podID="c52c2896-855d-422d-82db-504e4d38fec6" containerID="0e55f7f9c16819117bcd1e3241abdfd4234db25459a01614dfd0cd7704afbc49" exitCode=0 Jul 10 19:00:46.234541 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:46.234468 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vml84" event={"ID":"c52c2896-855d-422d-82db-504e4d38fec6","Type":"ContainerDied","Data":"0e55f7f9c16819117bcd1e3241abdfd4234db25459a01614dfd0cd7704afbc49"} Jul 10 19:00:46.235229 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:46.235216 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xm2hk" Jul 10 19:00:46.248770 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:46.248753 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:47.012655 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:47.012624 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:47.013159 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:47.012734 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9p2rq" podUID="f911a124-c338-40f4-983f-c61a0d899d93" Jul 10 19:00:47.013159 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:47.012792 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:47.013159 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:47.012890 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjklm" podUID="182da83f-63cb-4013-9868-4e95e297c58a" Jul 10 19:00:47.239888 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:47.239863 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:00:47.240219 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:47.240190 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" event={"ID":"f217ac86-726b-4051-a7b4-aea1fe005f49","Type":"ContainerStarted","Data":"08f25f4b5a80c00e29f267570975e93e105b094211120e397006830031f8e121"} Jul 10 19:00:47.240382 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:47.240363 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:47.240466 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:47.240434 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:47.255692 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:47.255671 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:00:47.306536 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:47.306492 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" podStartSLOduration=9.659380199 podStartE2EDuration="28.306479089s" totalImagesPullingTime="18.64709889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:00:47.305789307 +0000 UTC m=+28.842131733" watchObservedRunningTime="2026-07-10 19:00:47.306479089 +0000 UTC m=+28.842821520" Jul 10 19:00:47.953054 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:47.952845 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9p2rq"] Jul 10 19:00:47.953276 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:47.953112 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:47.953276 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:47.953206 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9p2rq" podUID="f911a124-c338-40f4-983f-c61a0d899d93" Jul 10 19:00:47.955114 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:47.955074 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4cdrm"] Jul 10 19:00:47.955233 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:47.955191 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:47.955311 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:47.955289 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4cdrm" podUID="a7f690e5-5f82-4573-bc6e-a7ed083f6fbc" Jul 10 19:00:47.958280 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:47.958260 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vjklm"] Jul 10 19:00:47.958384 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:47.958343 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:47.958450 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:47.958410 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjklm" podUID="182da83f-63cb-4013-9868-4e95e297c58a" Jul 10 19:00:48.244692 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:48.244615 2571 generic.go:356] "Generic (PLEG): container finished" podID="c52c2896-855d-422d-82db-504e4d38fec6" containerID="3e212b63993c804e91f8cdb833cd976223f6165c03a1725b072e6e37e866fb1c" exitCode=0 Jul 10 19:00:48.245187 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:48.244694 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vml84" event={"ID":"c52c2896-855d-422d-82db-504e4d38fec6","Type":"ContainerDied","Data":"3e212b63993c804e91f8cdb833cd976223f6165c03a1725b072e6e37e866fb1c"} Jul 10 19:00:50.012125 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:50.012047 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:50.012623 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:50.012161 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:50.012623 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:50.012165 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4cdrm" podUID="a7f690e5-5f82-4573-bc6e-a7ed083f6fbc" Jul 10 19:00:50.012623 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:50.012216 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:50.012623 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:50.012291 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjklm" podUID="182da83f-63cb-4013-9868-4e95e297c58a" Jul 10 19:00:50.012623 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:50.012364 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9p2rq" podUID="f911a124-c338-40f4-983f-c61a0d899d93" Jul 10 19:00:50.251832 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:50.251799 2571 generic.go:356] "Generic (PLEG): container finished" podID="c52c2896-855d-422d-82db-504e4d38fec6" containerID="d413ee150714dbb484b3fcacaf9a2944fdc3274e6ce96af23ca59432e0ffd67f" exitCode=0 Jul 10 19:00:50.251987 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:50.251858 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vml84" event={"ID":"c52c2896-855d-422d-82db-504e4d38fec6","Type":"ContainerDied","Data":"d413ee150714dbb484b3fcacaf9a2944fdc3274e6ce96af23ca59432e0ffd67f"} Jul 10 19:00:52.011925 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:52.011888 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:52.012399 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:52.011947 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:52.012399 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:52.011947 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:52.012399 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:52.012025 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjklm" podUID="182da83f-63cb-4013-9868-4e95e297c58a" Jul 10 19:00:52.012399 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:52.012158 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9p2rq" podUID="f911a124-c338-40f4-983f-c61a0d899d93" Jul 10 19:00:52.012399 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:52.012215 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4cdrm" podUID="a7f690e5-5f82-4573-bc6e-a7ed083f6fbc" Jul 10 19:00:52.748730 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:52.748517 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs\") pod \"network-metrics-daemon-9p2rq\" (UID: \"f911a124-c338-40f4-983f-c61a0d899d93\") " pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:52.748911 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:52.748767 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret\") pod \"global-pull-secret-syncer-4cdrm\" (UID: \"a7f690e5-5f82-4573-bc6e-a7ed083f6fbc\") " pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:52.748911 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:52.748675 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:00:52.748911 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:52.748859 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs podName:f911a124-c338-40f4-983f-c61a0d899d93 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:24.748838262 +0000 UTC m=+66.285180686 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs") pod "network-metrics-daemon-9p2rq" (UID: "f911a124-c338-40f4-983f-c61a0d899d93") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:00:52.748911 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:52.748895 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 19:00:52.749183 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:52.748942 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret podName:a7f690e5-5f82-4573-bc6e-a7ed083f6fbc nodeName:}" failed. No retries permitted until 2026-07-10 19:01:24.748927584 +0000 UTC m=+66.285270008 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret") pod "global-pull-secret-syncer-4cdrm" (UID: "a7f690e5-5f82-4573-bc6e-a7ed083f6fbc") : object "kube-system"/"original-pull-secret" not registered Jul 10 19:00:52.850132 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:52.850088 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8qhq\" (UniqueName: \"kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq\") pod \"network-check-target-vjklm\" (UID: \"182da83f-63cb-4013-9868-4e95e297c58a\") " pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:52.850289 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:52.850238 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 19:00:52.850289 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:52.850254 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 19:00:52.850289 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:52.850264 2571 projected.go:196] Error preparing data for projected volume kube-api-access-b8qhq for pod openshift-network-diagnostics/network-check-target-vjklm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:00:52.850387 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:52.850312 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq podName:182da83f-63cb-4013-9868-4e95e297c58a nodeName:}" failed. No retries permitted until 2026-07-10 19:01:24.850298121 +0000 UTC m=+66.386640536 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b8qhq" (UniqueName: "kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq") pod "network-check-target-vjklm" (UID: "182da83f-63cb-4013-9868-4e95e297c58a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:00:53.234484 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.234458 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeReady" Jul 10 19:00:53.234843 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.234599 2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 19:00:53.270310 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.270285 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-644b746fdc-rjsj4"] Jul 10 19:00:53.272461 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.272442 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.274982 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.274945 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 19:00:53.275085 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.275068 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 19:00:53.275183 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.275085 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 19:00:53.275183 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.275094 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-r62lv\"" Jul 10 19:00:53.282014 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.281993 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 19:00:53.283337 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.283319 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dptsw"] Jul 10 19:00:53.285416 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.285398 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dptsw" Jul 10 19:00:53.287057 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.287036 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-644b746fdc-rjsj4"] Jul 10 19:00:53.288000 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.287980 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gmlfz\"" Jul 10 19:00:53.288000 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.287996 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-47np5"] Jul 10 19:00:53.288262 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.288129 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 19:00:53.288262 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.288234 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 19:00:53.291366 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.291348 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-47np5" Jul 10 19:00:53.294151 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.294129 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 19:00:53.294250 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.294191 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 19:00:53.294250 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.294218 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 19:00:53.294359 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.294348 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jcvlx\"" Jul 10 19:00:53.299561 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.299542 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-47np5"] Jul 10 19:00:53.303568 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.302855 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dptsw"] Jul 10 19:00:53.455236 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.455147 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.455236 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.455193 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls\") pod \"dns-default-dptsw\" (UID: \"50e43ef8-5c11-457b-b37c-ca6711ecd556\") " pod="openshift-dns/dns-default-dptsw" Jul 10 19:00:53.455236 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.455219 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5501e15d-59e4-4a41-8cb0-77ea136593ca-trusted-ca\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.455544 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.455242 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5d26\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-kube-api-access-b5d26\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.455544 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.455271 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50e43ef8-5c11-457b-b37c-ca6711ecd556-tmp-dir\") pod \"dns-default-dptsw\" (UID: \"50e43ef8-5c11-457b-b37c-ca6711ecd556\") " pod="openshift-dns/dns-default-dptsw" Jul 10 19:00:53.455544 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.455290 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5501e15d-59e4-4a41-8cb0-77ea136593ca-ca-trust-extracted\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.455544 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.455324 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/50e43ef8-5c11-457b-b37c-ca6711ecd556-config-volume\") pod \"dns-default-dptsw\" (UID: \"50e43ef8-5c11-457b-b37c-ca6711ecd556\") " pod="openshift-dns/dns-default-dptsw" Jul 10 19:00:53.455544 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.455345 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1bb26a4-858a-4c24-86ac-ef53647ded74-cert\") pod \"ingress-canary-47np5\" (UID: \"a1bb26a4-858a-4c24-86ac-ef53647ded74\") " pod="openshift-ingress-canary/ingress-canary-47np5" Jul 10 19:00:53.455544 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.455391 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5501e15d-59e4-4a41-8cb0-77ea136593ca-image-registry-private-configuration\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.455544 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.455415 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-certificates\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.455544 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.455430 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5501e15d-59e4-4a41-8cb0-77ea136593ca-installation-pull-secrets\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.455544 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.455453 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-bound-sa-token\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.455544 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.455484 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fwn9\" (UniqueName: \"kubernetes.io/projected/a1bb26a4-858a-4c24-86ac-ef53647ded74-kube-api-access-4fwn9\") pod \"ingress-canary-47np5\" (UID: \"a1bb26a4-858a-4c24-86ac-ef53647ded74\") " pod="openshift-ingress-canary/ingress-canary-47np5" Jul 10 19:00:53.455544 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.455521 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjvx7\" (UniqueName: \"kubernetes.io/projected/50e43ef8-5c11-457b-b37c-ca6711ecd556-kube-api-access-wjvx7\") pod \"dns-default-dptsw\" (UID: \"50e43ef8-5c11-457b-b37c-ca6711ecd556\") " pod="openshift-dns/dns-default-dptsw" Jul 10 19:00:53.556056 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.556015 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls\") pod \"dns-default-dptsw\" (UID: \"50e43ef8-5c11-457b-b37c-ca6711ecd556\") " pod="openshift-dns/dns-default-dptsw" Jul 10 19:00:53.556056 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.556061 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5501e15d-59e4-4a41-8cb0-77ea136593ca-trusted-ca\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.556291 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:53.556202 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 19:00:53.556291 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.556225 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5d26\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-kube-api-access-b5d26\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.556291 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.556283 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50e43ef8-5c11-457b-b37c-ca6711ecd556-tmp-dir\") pod \"dns-default-dptsw\" (UID: \"50e43ef8-5c11-457b-b37c-ca6711ecd556\") " pod="openshift-dns/dns-default-dptsw" Jul 10 19:00:53.556420 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.556323 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5501e15d-59e4-4a41-8cb0-77ea136593ca-ca-trust-extracted\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.556420 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.556375 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/50e43ef8-5c11-457b-b37c-ca6711ecd556-config-volume\") pod \"dns-default-dptsw\" (UID: \"50e43ef8-5c11-457b-b37c-ca6711ecd556\") " pod="openshift-dns/dns-default-dptsw" Jul 10 19:00:53.556420 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.556406 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1bb26a4-858a-4c24-86ac-ef53647ded74-cert\") pod \"ingress-canary-47np5\" (UID: \"a1bb26a4-858a-4c24-86ac-ef53647ded74\") " pod="openshift-ingress-canary/ingress-canary-47np5" Jul 10 19:00:53.556544 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.556476 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5501e15d-59e4-4a41-8cb0-77ea136593ca-image-registry-private-configuration\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.556544 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.556509 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-certificates\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.556544 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.556538 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5501e15d-59e4-4a41-8cb0-77ea136593ca-installation-pull-secrets\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.556653 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.556573 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-bound-sa-token\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.556653 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.556637 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fwn9\" (UniqueName: \"kubernetes.io/projected/a1bb26a4-858a-4c24-86ac-ef53647ded74-kube-api-access-4fwn9\") pod \"ingress-canary-47np5\" (UID: \"a1bb26a4-858a-4c24-86ac-ef53647ded74\") " pod="openshift-ingress-canary/ingress-canary-47np5" Jul 10 19:00:53.556944 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.556917 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjvx7\" (UniqueName: \"kubernetes.io/projected/50e43ef8-5c11-457b-b37c-ca6711ecd556-kube-api-access-wjvx7\") pod \"dns-default-dptsw\" (UID: \"50e43ef8-5c11-457b-b37c-ca6711ecd556\") " pod="openshift-dns/dns-default-dptsw" Jul 10 19:00:53.557031 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.556982 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.557137 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:53.557119 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 19:00:53.557352 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:53.557333 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644b746fdc-rjsj4: secret "image-registry-tls" not found Jul 10 19:00:53.557435 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:53.557424 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls podName:5501e15d-59e4-4a41-8cb0-77ea136593ca nodeName:}" failed. No retries permitted until 2026-07-10 19:00:54.057394921 +0000 UTC m=+35.593737344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls") pod "image-registry-644b746fdc-rjsj4" (UID: "5501e15d-59e4-4a41-8cb0-77ea136593ca") : secret "image-registry-tls" not found Jul 10 19:00:53.557505 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.556920 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5501e15d-59e4-4a41-8cb0-77ea136593ca-ca-trust-extracted\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.557865 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.557838 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5501e15d-59e4-4a41-8cb0-77ea136593ca-trusted-ca\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.557970 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.557908 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50e43ef8-5c11-457b-b37c-ca6711ecd556-tmp-dir\") pod \"dns-default-dptsw\" (UID: \"50e43ef8-5c11-457b-b37c-ca6711ecd556\") " pod="openshift-dns/dns-default-dptsw" Jul 10 19:00:53.558023 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.557992 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/50e43ef8-5c11-457b-b37c-ca6711ecd556-config-volume\") pod \"dns-default-dptsw\" (UID: \"50e43ef8-5c11-457b-b37c-ca6711ecd556\") " pod="openshift-dns/dns-default-dptsw" Jul 10 19:00:53.558067 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:53.558054 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls podName:50e43ef8-5c11-457b-b37c-ca6711ecd556 nodeName:}" failed. No retries permitted until 2026-07-10 19:00:54.058026105 +0000 UTC m=+35.594368522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls") pod "dns-default-dptsw" (UID: "50e43ef8-5c11-457b-b37c-ca6711ecd556") : secret "dns-default-metrics-tls" not found Jul 10 19:00:53.558540 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.558346 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-certificates\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.562256 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.561848 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5501e15d-59e4-4a41-8cb0-77ea136593ca-installation-pull-secrets\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.562256 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.561896 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1bb26a4-858a-4c24-86ac-ef53647ded74-cert\") pod \"ingress-canary-47np5\" (UID: \"a1bb26a4-858a-4c24-86ac-ef53647ded74\") " pod="openshift-ingress-canary/ingress-canary-47np5" Jul 10 19:00:53.563911 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.563893 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5501e15d-59e4-4a41-8cb0-77ea136593ca-image-registry-private-configuration\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.566381 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.566336 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fwn9\" (UniqueName: \"kubernetes.io/projected/a1bb26a4-858a-4c24-86ac-ef53647ded74-kube-api-access-4fwn9\") pod \"ingress-canary-47np5\" (UID: \"a1bb26a4-858a-4c24-86ac-ef53647ded74\") " pod="openshift-ingress-canary/ingress-canary-47np5" Jul 10 19:00:53.566478 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.566436 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5d26\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-kube-api-access-b5d26\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.567188 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.567166 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjvx7\" (UniqueName: \"kubernetes.io/projected/50e43ef8-5c11-457b-b37c-ca6711ecd556-kube-api-access-wjvx7\") pod \"dns-default-dptsw\" (UID: \"50e43ef8-5c11-457b-b37c-ca6711ecd556\") " pod="openshift-dns/dns-default-dptsw" Jul 10 19:00:53.567480 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.567439 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-bound-sa-token\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:53.605131 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:53.605091 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-47np5" Jul 10 19:00:54.012388 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:54.012357 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:00:54.012388 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:54.012379 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:00:54.012618 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:54.012382 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:00:54.016666 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:54.016635 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 19:00:54.016801 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:54.016724 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 19:00:54.016801 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:54.016734 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zb6nn\"" Jul 10 19:00:54.016801 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:54.016737 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 19:00:54.017004 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:54.016748 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 19:00:54.017004 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:54.016843 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sqhv9\"" Jul 10 19:00:54.062085 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:54.062052 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:54.062209 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:54.062125 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls\") pod \"dns-default-dptsw\" (UID: \"50e43ef8-5c11-457b-b37c-ca6711ecd556\") " pod="openshift-dns/dns-default-dptsw" Jul 10 19:00:54.062209 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:54.062191 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 19:00:54.062306 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:54.062211 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644b746fdc-rjsj4: secret "image-registry-tls" not found Jul 10 19:00:54.062306 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:54.062249 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 19:00:54.062306 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:54.062279 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls podName:5501e15d-59e4-4a41-8cb0-77ea136593ca nodeName:}" failed. No retries permitted until 2026-07-10 19:00:55.062263869 +0000 UTC m=+36.598606280 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls") pod "image-registry-644b746fdc-rjsj4" (UID: "5501e15d-59e4-4a41-8cb0-77ea136593ca") : secret "image-registry-tls" not found Jul 10 19:00:54.062306 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:54.062303 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls podName:50e43ef8-5c11-457b-b37c-ca6711ecd556 nodeName:}" failed. No retries permitted until 2026-07-10 19:00:55.062287761 +0000 UTC m=+36.598630182 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls") pod "dns-default-dptsw" (UID: "50e43ef8-5c11-457b-b37c-ca6711ecd556") : secret "dns-default-metrics-tls" not found Jul 10 19:00:55.007826 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:55.007613 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-47np5"] Jul 10 19:00:55.011402 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:55.011374 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1bb26a4_858a_4c24_86ac_ef53647ded74.slice/crio-69e9558e6a40c00019a335527fb75b98504e3b56a24e085f1e45b6207435af8b WatchSource:0}: Error finding container 69e9558e6a40c00019a335527fb75b98504e3b56a24e085f1e45b6207435af8b: Status 404 returned error can't find the container with id 69e9558e6a40c00019a335527fb75b98504e3b56a24e085f1e45b6207435af8b Jul 10 19:00:55.072146 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:55.072118 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:55.072265 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:55.072167 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls\") pod \"dns-default-dptsw\" (UID: \"50e43ef8-5c11-457b-b37c-ca6711ecd556\") " pod="openshift-dns/dns-default-dptsw" Jul 10 19:00:55.072265 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:55.072211 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 19:00:55.072265 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:55.072227 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644b746fdc-rjsj4: secret "image-registry-tls" not found Jul 10 19:00:55.072265 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:55.072252 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 19:00:55.072407 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:55.072299 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls podName:50e43ef8-5c11-457b-b37c-ca6711ecd556 nodeName:}" failed. No retries permitted until 2026-07-10 19:00:57.07228246 +0000 UTC m=+38.608624882 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls") pod "dns-default-dptsw" (UID: "50e43ef8-5c11-457b-b37c-ca6711ecd556") : secret "dns-default-metrics-tls" not found Jul 10 19:00:55.072407 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:55.072314 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls podName:5501e15d-59e4-4a41-8cb0-77ea136593ca nodeName:}" failed. No retries permitted until 2026-07-10 19:00:57.072306347 +0000 UTC m=+38.608648757 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls") pod "image-registry-644b746fdc-rjsj4" (UID: "5501e15d-59e4-4a41-8cb0-77ea136593ca") : secret "image-registry-tls" not found Jul 10 19:00:55.266305 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:55.266221 2571 generic.go:356] "Generic (PLEG): container finished" podID="c52c2896-855d-422d-82db-504e4d38fec6" containerID="df4698efa81bf49d34399efc94f9ba238d7df16cf39b663c4009bf208f9ec6aa" exitCode=0 Jul 10 19:00:55.266443 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:55.266294 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vml84" event={"ID":"c52c2896-855d-422d-82db-504e4d38fec6","Type":"ContainerDied","Data":"df4698efa81bf49d34399efc94f9ba238d7df16cf39b663c4009bf208f9ec6aa"} Jul 10 19:00:55.267390 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:55.267369 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-47np5" event={"ID":"a1bb26a4-858a-4c24-86ac-ef53647ded74","Type":"ContainerStarted","Data":"69e9558e6a40c00019a335527fb75b98504e3b56a24e085f1e45b6207435af8b"} Jul 10 19:00:56.273564 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:56.273524 2571 generic.go:356] "Generic (PLEG): container finished" podID="c52c2896-855d-422d-82db-504e4d38fec6" containerID="b4f268f24533b22973934add304e4e92176c9edc56406ac17d87a96b249c03e5" exitCode=0 Jul 10 19:00:56.274079 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:56.273584 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vml84" event={"ID":"c52c2896-855d-422d-82db-504e4d38fec6","Type":"ContainerDied","Data":"b4f268f24533b22973934add304e4e92176c9edc56406ac17d87a96b249c03e5"} Jul 10 19:00:57.036724 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.036692 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls"] Jul 10 19:00:57.039566 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.039542 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" Jul 10 19:00:57.039731 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.039673 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-68c78766fd-2z8xb"] Jul 10 19:00:57.041492 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.041471 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-7h2bn"] Jul 10 19:00:57.041622 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.041603 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:00:57.042285 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.042264 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 10 19:00:57.042391 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.042365 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 19:00:57.042450 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.042365 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 19:00:57.042534 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.042511 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-hb7cz\"" Jul 10 19:00:57.043357 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.043339 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.044063 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.044046 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 19:00:57.044063 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.044056 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 10 19:00:57.044225 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.044061 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 10 19:00:57.045669 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.045508 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 10 19:00:57.045669 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.045531 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-mb4h5\"" Jul 10 19:00:57.045669 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.045556 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 19:00:57.045669 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.045568 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 10 19:00:57.045669 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.045592 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 10 19:00:57.045669 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.045536 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 10 19:00:57.046051 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.045884 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 19:00:57.046229 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.046213 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 19:00:57.046316 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.046258 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-sgnzp\"" Jul 10 19:00:57.046456 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.046437 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 10 19:00:57.051684 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.051667 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls"] Jul 10 19:00:57.052357 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.052329 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 10 19:00:57.056185 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.056167 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-7h2bn"] Jul 10 19:00:57.064570 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.064549 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68c78766fd-2z8xb"] Jul 10 19:00:57.089132 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.088788 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:00:57.089132 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.088817 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls\") pod \"dns-default-dptsw\" (UID: \"50e43ef8-5c11-457b-b37c-ca6711ecd556\") " pod="openshift-dns/dns-default-dptsw" Jul 10 19:00:57.089132 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:57.088938 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 19:00:57.089132 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:57.088986 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls podName:50e43ef8-5c11-457b-b37c-ca6711ecd556 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:01.088969624 +0000 UTC m=+42.625312040 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls") pod "dns-default-dptsw" (UID: "50e43ef8-5c11-457b-b37c-ca6711ecd556") : secret "dns-default-metrics-tls" not found Jul 10 19:00:57.089419 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:57.089210 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 19:00:57.089419 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:57.089226 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644b746fdc-rjsj4: secret "image-registry-tls" not found Jul 10 19:00:57.089419 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:57.089284 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls podName:5501e15d-59e4-4a41-8cb0-77ea136593ca nodeName:}" failed. No retries permitted until 2026-07-10 19:01:01.089267565 +0000 UTC m=+42.625609984 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls") pod "image-registry-644b746fdc-rjsj4" (UID: "5501e15d-59e4-4a41-8cb0-77ea136593ca") : secret "image-registry-tls" not found Jul 10 19:00:57.189729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.189615 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:00:57.189729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.189669 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3af4a62b-153c-4bc5-82f6-722a7f1e5259-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-m7vls\" (UID: \"3af4a62b-153c-4bc5-82f6-722a7f1e5259\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" Jul 10 19:00:57.189729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.189693 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nngqn\" (UniqueName: \"kubernetes.io/projected/3af4a62b-153c-4bc5-82f6-722a7f1e5259-kube-api-access-nngqn\") pod \"cluster-monitoring-operator-5f7f494dd4-m7vls\" (UID: \"3af4a62b-153c-4bc5-82f6-722a7f1e5259\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" Jul 10 19:00:57.189729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.189715 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ed7ba8a-9eab-45ed-944c-fd0c5d905551-tmp\") pod \"insights-operator-68d9dd94cf-7h2bn\" (UID: \"6ed7ba8a-9eab-45ed-944c-fd0c5d905551\") " pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.189729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.189737 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blm2j\" (UniqueName: \"kubernetes.io/projected/6ed7ba8a-9eab-45ed-944c-fd0c5d905551-kube-api-access-blm2j\") pod \"insights-operator-68d9dd94cf-7h2bn\" (UID: \"6ed7ba8a-9eab-45ed-944c-fd0c5d905551\") " pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.190215 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.189851 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-stats-auth\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:00:57.190215 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.189915 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgm4c\" (UniqueName: \"kubernetes.io/projected/97aba93c-11ba-49b1-8d28-3e26078983b2-kube-api-access-lgm4c\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:00:57.190215 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.189946 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-default-certificate\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:00:57.190215 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.190005 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed7ba8a-9eab-45ed-944c-fd0c5d905551-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-7h2bn\" (UID: \"6ed7ba8a-9eab-45ed-944c-fd0c5d905551\") " pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.190215 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.190033 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-m7vls\" (UID: \"3af4a62b-153c-4bc5-82f6-722a7f1e5259\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" Jul 10 19:00:57.190215 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.190058 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed7ba8a-9eab-45ed-944c-fd0c5d905551-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-7h2bn\" (UID: \"6ed7ba8a-9eab-45ed-944c-fd0c5d905551\") " pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.190215 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.190085 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6ed7ba8a-9eab-45ed-944c-fd0c5d905551-serving-cert\") pod \"insights-operator-68d9dd94cf-7h2bn\" (UID: \"6ed7ba8a-9eab-45ed-944c-fd0c5d905551\") " pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.190215 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.190161 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:00:57.190215 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.190200 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6ed7ba8a-9eab-45ed-944c-fd0c5d905551-snapshots\") pod \"insights-operator-68d9dd94cf-7h2bn\" (UID: \"6ed7ba8a-9eab-45ed-944c-fd0c5d905551\") " pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.279901 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.279869 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vml84" event={"ID":"c52c2896-855d-422d-82db-504e4d38fec6","Type":"ContainerStarted","Data":"2c3175944174c044df6f08d5c30de3873e22395308e5ad89c318b070b9bf6999"} Jul 10 19:00:57.291680 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.291612 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:00:57.291680 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.291653 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3af4a62b-153c-4bc5-82f6-722a7f1e5259-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-m7vls\" (UID: \"3af4a62b-153c-4bc5-82f6-722a7f1e5259\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" Jul 10 19:00:57.291680 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.291671 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nngqn\" (UniqueName: \"kubernetes.io/projected/3af4a62b-153c-4bc5-82f6-722a7f1e5259-kube-api-access-nngqn\") pod \"cluster-monitoring-operator-5f7f494dd4-m7vls\" (UID: \"3af4a62b-153c-4bc5-82f6-722a7f1e5259\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" Jul 10 19:00:57.291896 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.291690 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ed7ba8a-9eab-45ed-944c-fd0c5d905551-tmp\") pod \"insights-operator-68d9dd94cf-7h2bn\" (UID: \"6ed7ba8a-9eab-45ed-944c-fd0c5d905551\") " pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.291896 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.291709 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blm2j\" (UniqueName: \"kubernetes.io/projected/6ed7ba8a-9eab-45ed-944c-fd0c5d905551-kube-api-access-blm2j\") pod \"insights-operator-68d9dd94cf-7h2bn\" (UID: \"6ed7ba8a-9eab-45ed-944c-fd0c5d905551\") " pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.291896 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.291746 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-stats-auth\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:00:57.291896 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:57.291756 2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 19:00:57.291896 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.291774 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgm4c\" (UniqueName: \"kubernetes.io/projected/97aba93c-11ba-49b1-8d28-3e26078983b2-kube-api-access-lgm4c\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:00:57.291896 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.291792 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-default-certificate\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:00:57.291896 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:57.291818 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs podName:97aba93c-11ba-49b1-8d28-3e26078983b2 nodeName:}" failed. No retries permitted until 2026-07-10 19:00:57.791797333 +0000 UTC m=+39.328139744 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs") pod "router-default-68c78766fd-2z8xb" (UID: "97aba93c-11ba-49b1-8d28-3e26078983b2") : secret "router-metrics-certs-default" not found Jul 10 19:00:57.291896 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.291892 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed7ba8a-9eab-45ed-944c-fd0c5d905551-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-7h2bn\" (UID: \"6ed7ba8a-9eab-45ed-944c-fd0c5d905551\") " pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.292312 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.291926 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-m7vls\" (UID: \"3af4a62b-153c-4bc5-82f6-722a7f1e5259\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" Jul 10 19:00:57.292312 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.291955 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed7ba8a-9eab-45ed-944c-fd0c5d905551-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-7h2bn\" (UID: \"6ed7ba8a-9eab-45ed-944c-fd0c5d905551\") " pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.292312 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.291979 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6ed7ba8a-9eab-45ed-944c-fd0c5d905551-serving-cert\") pod \"insights-operator-68d9dd94cf-7h2bn\" (UID: \"6ed7ba8a-9eab-45ed-944c-fd0c5d905551\") " pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.292312 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.292009 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:00:57.292312 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.292036 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6ed7ba8a-9eab-45ed-944c-fd0c5d905551-snapshots\") pod \"insights-operator-68d9dd94cf-7h2bn\" (UID: \"6ed7ba8a-9eab-45ed-944c-fd0c5d905551\") " pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.292312 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.292080 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ed7ba8a-9eab-45ed-944c-fd0c5d905551-tmp\") pod \"insights-operator-68d9dd94cf-7h2bn\" (UID: \"6ed7ba8a-9eab-45ed-944c-fd0c5d905551\") " pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.292312 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:57.292159 2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 19:00:57.292312 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:57.292214 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle podName:97aba93c-11ba-49b1-8d28-3e26078983b2 nodeName:}" failed. No retries permitted until 2026-07-10 19:00:57.792190831 +0000 UTC m=+39.328533246 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle") pod "router-default-68c78766fd-2z8xb" (UID: "97aba93c-11ba-49b1-8d28-3e26078983b2") : configmap references non-existent config key: service-ca.crt Jul 10 19:00:57.292312 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:57.292236 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls podName:3af4a62b-153c-4bc5-82f6-722a7f1e5259 nodeName:}" failed. No retries permitted until 2026-07-10 19:00:57.792224076 +0000 UTC m=+39.328566487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-m7vls" (UID: "3af4a62b-153c-4bc5-82f6-722a7f1e5259") : secret "cluster-monitoring-operator-tls" not found Jul 10 19:00:57.292757 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.292502 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3af4a62b-153c-4bc5-82f6-722a7f1e5259-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-m7vls\" (UID: \"3af4a62b-153c-4bc5-82f6-722a7f1e5259\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" Jul 10 19:00:57.292757 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.292651 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6ed7ba8a-9eab-45ed-944c-fd0c5d905551-snapshots\") pod \"insights-operator-68d9dd94cf-7h2bn\" (UID: \"6ed7ba8a-9eab-45ed-944c-fd0c5d905551\") " pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.293292 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.293267 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed7ba8a-9eab-45ed-944c-fd0c5d905551-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-7h2bn\" (UID: \"6ed7ba8a-9eab-45ed-944c-fd0c5d905551\") " pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.293650 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.293626 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed7ba8a-9eab-45ed-944c-fd0c5d905551-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-7h2bn\" (UID: \"6ed7ba8a-9eab-45ed-944c-fd0c5d905551\") " pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.295536 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.295518 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6ed7ba8a-9eab-45ed-944c-fd0c5d905551-serving-cert\") pod \"insights-operator-68d9dd94cf-7h2bn\" (UID: \"6ed7ba8a-9eab-45ed-944c-fd0c5d905551\") " pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.295807 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.295781 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-default-certificate\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:00:57.295877 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.295790 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-stats-auth\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:00:57.304228 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.304194 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nngqn\" (UniqueName: \"kubernetes.io/projected/3af4a62b-153c-4bc5-82f6-722a7f1e5259-kube-api-access-nngqn\") pod \"cluster-monitoring-operator-5f7f494dd4-m7vls\" (UID: \"3af4a62b-153c-4bc5-82f6-722a7f1e5259\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" Jul 10 19:00:57.305273 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.305244 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgm4c\" (UniqueName: \"kubernetes.io/projected/97aba93c-11ba-49b1-8d28-3e26078983b2-kube-api-access-lgm4c\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:00:57.306248 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.306218 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blm2j\" (UniqueName: \"kubernetes.io/projected/6ed7ba8a-9eab-45ed-944c-fd0c5d905551-kube-api-access-blm2j\") pod \"insights-operator-68d9dd94cf-7h2bn\" (UID: \"6ed7ba8a-9eab-45ed-944c-fd0c5d905551\") " pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.309251 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.309187 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vml84" podStartSLOduration=8.291366666 podStartE2EDuration="38.309169589s" totalImagesPullingTime="30.017802923s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:00:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:00:57.308744533 +0000 UTC m=+38.845086975" watchObservedRunningTime="2026-07-10 19:00:57.309169589 +0000 UTC m=+38.845512022" Jul 10 19:00:57.369383 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.369356 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" Jul 10 19:00:57.701984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.701837 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-7h2bn"] Jul 10 19:00:57.704435 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:00:57.704407 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ed7ba8a_9eab_45ed_944c_fd0c5d905551.slice/crio-96756ff39b597287a92de0536aedd80e29575199e4871a385b59fdf95f5d87b0 WatchSource:0}: Error finding container 96756ff39b597287a92de0536aedd80e29575199e4871a385b59fdf95f5d87b0: Status 404 returned error can't find the container with id 96756ff39b597287a92de0536aedd80e29575199e4871a385b59fdf95f5d87b0 Jul 10 19:00:57.797302 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.797265 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-m7vls\" (UID: \"3af4a62b-153c-4bc5-82f6-722a7f1e5259\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" Jul 10 19:00:57.797302 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.797302 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:00:57.797482 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:57.797414 2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 19:00:57.797482 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:57.797425 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle podName:97aba93c-11ba-49b1-8d28-3e26078983b2 nodeName:}" failed. No retries permitted until 2026-07-10 19:00:58.797410402 +0000 UTC m=+40.333752812 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle") pod "router-default-68c78766fd-2z8xb" (UID: "97aba93c-11ba-49b1-8d28-3e26078983b2") : configmap references non-existent config key: service-ca.crt Jul 10 19:00:57.797576 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:57.797491 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:00:57.797576 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:57.797531 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls podName:3af4a62b-153c-4bc5-82f6-722a7f1e5259 nodeName:}" failed. No retries permitted until 2026-07-10 19:00:58.797509479 +0000 UTC m=+40.333851895 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-m7vls" (UID: "3af4a62b-153c-4bc5-82f6-722a7f1e5259") : secret "cluster-monitoring-operator-tls" not found Jul 10 19:00:57.797576 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:57.797563 2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 19:00:57.797682 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:57.797605 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs podName:97aba93c-11ba-49b1-8d28-3e26078983b2 nodeName:}" failed. No retries permitted until 2026-07-10 19:00:58.79759394 +0000 UTC m=+40.333936349 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs") pod "router-default-68c78766fd-2z8xb" (UID: "97aba93c-11ba-49b1-8d28-3e26078983b2") : secret "router-metrics-certs-default" not found Jul 10 19:00:58.282846 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:58.282812 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" event={"ID":"6ed7ba8a-9eab-45ed-944c-fd0c5d905551","Type":"ContainerStarted","Data":"96756ff39b597287a92de0536aedd80e29575199e4871a385b59fdf95f5d87b0"} Jul 10 19:00:58.284306 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:58.284283 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-47np5" event={"ID":"a1bb26a4-858a-4c24-86ac-ef53647ded74","Type":"ContainerStarted","Data":"62efb5d92252ad45aa66ab97dbeb9dd19144d52bbadd93194084bd5b7340dd99"} Jul 10 19:00:58.302832 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:58.302774 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-47np5" podStartSLOduration=2.391984547 podStartE2EDuration="5.3027553s" totalImagesPullingTime="2.910770753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:00:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:00:58.302730664 +0000 UTC m=+39.839073096" watchObservedRunningTime="2026-07-10 19:00:58.3027553 +0000 UTC m=+39.839097735" Jul 10 19:00:58.807081 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:58.807040 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-m7vls\" (UID: \"3af4a62b-153c-4bc5-82f6-722a7f1e5259\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" Jul 10 19:00:58.807281 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:58.807118 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:00:58.807281 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:00:58.807200 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:00:58.807281 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:58.807221 2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 19:00:58.807430 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:58.807287 2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 19:00:58.807430 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:58.807295 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls podName:3af4a62b-153c-4bc5-82f6-722a7f1e5259 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:00.807275202 +0000 UTC m=+42.343617627 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-m7vls" (UID: "3af4a62b-153c-4bc5-82f6-722a7f1e5259") : secret "cluster-monitoring-operator-tls" not found Jul 10 19:00:58.807430 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:58.807351 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle podName:97aba93c-11ba-49b1-8d28-3e26078983b2 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:00.807334623 +0000 UTC m=+42.343677041 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle") pod "router-default-68c78766fd-2z8xb" (UID: "97aba93c-11ba-49b1-8d28-3e26078983b2") : configmap references non-existent config key: service-ca.crt Jul 10 19:00:58.807430 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:00:58.807367 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs podName:97aba93c-11ba-49b1-8d28-3e26078983b2 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:00.807359119 +0000 UTC m=+42.343701529 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs") pod "router-default-68c78766fd-2z8xb" (UID: "97aba93c-11ba-49b1-8d28-3e26078983b2") : secret "router-metrics-certs-default" not found Jul 10 19:01:00.827249 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:00.824330 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:01:00.827249 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:00.824478 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-m7vls\" (UID: \"3af4a62b-153c-4bc5-82f6-722a7f1e5259\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" Jul 10 19:01:00.827249 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:00.824532 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:01:00.827249 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:00.824692 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle podName:97aba93c-11ba-49b1-8d28-3e26078983b2 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:04.82467268 +0000 UTC m=+46.361015110 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle") pod "router-default-68c78766fd-2z8xb" (UID: "97aba93c-11ba-49b1-8d28-3e26078983b2") : configmap references non-existent config key: service-ca.crt Jul 10 19:01:00.827249 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:00.824899 2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 19:01:00.827249 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:00.824965 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls podName:3af4a62b-153c-4bc5-82f6-722a7f1e5259 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:04.824948692 +0000 UTC m=+46.361291107 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-m7vls" (UID: "3af4a62b-153c-4bc5-82f6-722a7f1e5259") : secret "cluster-monitoring-operator-tls" not found Jul 10 19:01:00.827249 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:00.825042 2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 19:01:00.827249 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:00.825081 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs podName:97aba93c-11ba-49b1-8d28-3e26078983b2 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:04.82506303 +0000 UTC m=+46.361405445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs") pod "router-default-68c78766fd-2z8xb" (UID: "97aba93c-11ba-49b1-8d28-3e26078983b2") : secret "router-metrics-certs-default" not found Jul 10 19:01:01.126907 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.126808 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:01:01.126907 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.126852 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls\") pod \"dns-default-dptsw\" (UID: \"50e43ef8-5c11-457b-b37c-ca6711ecd556\") " pod="openshift-dns/dns-default-dptsw" Jul 10 19:01:01.127140 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:01.126979 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 19:01:01.127140 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:01.126989 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 19:01:01.127140 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:01.127012 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644b746fdc-rjsj4: secret "image-registry-tls" not found Jul 10 19:01:01.127140 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:01.127041 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls podName:50e43ef8-5c11-457b-b37c-ca6711ecd556 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:09.127024963 +0000 UTC m=+50.663367396 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls") pod "dns-default-dptsw" (UID: "50e43ef8-5c11-457b-b37c-ca6711ecd556") : secret "dns-default-metrics-tls" not found Jul 10 19:01:01.127140 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:01.127077 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls podName:5501e15d-59e4-4a41-8cb0-77ea136593ca nodeName:}" failed. No retries permitted until 2026-07-10 19:01:09.127060401 +0000 UTC m=+50.663402813 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls") pod "image-registry-644b746fdc-rjsj4" (UID: "5501e15d-59e4-4a41-8cb0-77ea136593ca") : secret "image-registry-tls" not found Jul 10 19:01:01.224657 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.224624 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dn2cb"] Jul 10 19:01:01.239947 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.239924 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:01.240678 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.240647 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dn2cb"] Jul 10 19:01:01.243797 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.243772 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f67rl\"" Jul 10 19:01:01.244362 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.244342 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 19:01:01.244649 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.244636 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 19:01:01.291937 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.291910 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" event={"ID":"6ed7ba8a-9eab-45ed-944c-fd0c5d905551","Type":"ContainerStarted","Data":"45b1290e8b898f545732e82ebc71119ff46baa87159ec05afae46077847f0e23"} Jul 10 19:01:01.328956 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.328930 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:01.329079 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.328968 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-crio-socket\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:01.329079 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.329001 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:01.329273 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.329241 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mglc4\" (UniqueName: \"kubernetes.io/projected/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-api-access-mglc4\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:01.329417 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.329345 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-data-volume\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:01.430693 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.430623 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-crio-socket\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:01.430693 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.430673 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:01.430857 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.430773 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mglc4\" (UniqueName: \"kubernetes.io/projected/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-api-access-mglc4\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:01.430857 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.430833 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-data-volume\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:01.430857 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:01.430836 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 19:01:01.430857 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.430848 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-crio-socket\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:01.431044 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.430861 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:01.431044 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:01.430917 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-rbac-proxy-cm podName:92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:01.930895985 +0000 UTC m=+43.467238416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dn2cb" (UID: "92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10") : configmap "kube-rbac-proxy" not found Jul 10 19:01:01.431044 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:01.430933 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 19:01:01.431044 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:01.430984 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls podName:92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:01.930971203 +0000 UTC m=+43.467313628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dn2cb" (UID: "92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10") : secret "insights-runtime-extractor-tls" not found Jul 10 19:01:01.431223 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.431174 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-data-volume\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:01.440060 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.440040 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mglc4\" (UniqueName: \"kubernetes.io/projected/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-api-access-mglc4\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:01.934758 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.934724 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:01.935167 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:01.934770 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:01.935167 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:01.934879 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 19:01:01.935167 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:01.934939 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls podName:92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:02.934922162 +0000 UTC m=+44.471264572 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dn2cb" (UID: "92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10") : secret "insights-runtime-extractor-tls" not found Jul 10 19:01:01.935167 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:01.934885 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 19:01:01.935167 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:01.935001 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-rbac-proxy-cm podName:92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:02.93498935 +0000 UTC m=+44.471331761 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dn2cb" (UID: "92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10") : configmap "kube-rbac-proxy" not found Jul 10 19:01:02.942995 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:02.942961 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:02.943369 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:02.943082 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 19:01:02.943369 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:02.943091 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:02.943369 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:02.943155 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-rbac-proxy-cm podName:92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:04.943139685 +0000 UTC m=+46.479482095 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dn2cb" (UID: "92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10") : configmap "kube-rbac-proxy" not found Jul 10 19:01:02.943369 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:02.943208 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 19:01:02.943369 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:02.943263 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls podName:92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:04.943248321 +0000 UTC m=+46.479590732 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dn2cb" (UID: "92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10") : secret "insights-runtime-extractor-tls" not found Jul 10 19:01:03.022469 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:03.022444 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vrqsx_18f374ad-202d-43e8-b083-aabf526ff3d7/dns-node-resolver/0.log" Jul 10 19:01:03.620883 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:03.620859 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hzqbp_965c2d5d-ff87-46d2-9de2-7062e810316f/node-ca/0.log" Jul 10 19:01:04.221128 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:04.221074 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-47np5_a1bb26a4-858a-4c24-86ac-ef53647ded74/serve-healthcheck-canary/0.log" Jul 10 19:01:04.858881 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:04.858846 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:01:04.859076 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:04.858970 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-m7vls\" (UID: \"3af4a62b-153c-4bc5-82f6-722a7f1e5259\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" Jul 10 19:01:04.859076 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:04.859001 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:01:04.859076 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:04.859001 2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 19:01:04.859273 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:04.859039 2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 19:01:04.859273 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:04.859133 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs podName:97aba93c-11ba-49b1-8d28-3e26078983b2 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:12.859094072 +0000 UTC m=+54.395436498 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs") pod "router-default-68c78766fd-2z8xb" (UID: "97aba93c-11ba-49b1-8d28-3e26078983b2") : secret "router-metrics-certs-default" not found Jul 10 19:01:04.859273 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:04.859158 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls podName:3af4a62b-153c-4bc5-82f6-722a7f1e5259 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:12.859142959 +0000 UTC m=+54.395485373 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-m7vls" (UID: "3af4a62b-153c-4bc5-82f6-722a7f1e5259") : secret "cluster-monitoring-operator-tls" not found Jul 10 19:01:04.859273 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:04.859174 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle podName:97aba93c-11ba-49b1-8d28-3e26078983b2 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:12.859166445 +0000 UTC m=+54.395508855 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle") pod "router-default-68c78766fd-2z8xb" (UID: "97aba93c-11ba-49b1-8d28-3e26078983b2") : configmap references non-existent config key: service-ca.crt Jul 10 19:01:04.959959 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:04.959927 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:04.960158 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:04.959986 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:04.960158 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:04.960062 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 19:01:04.960158 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:04.960118 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 19:01:04.960158 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:04.960140 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls podName:92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:08.960123778 +0000 UTC m=+50.496466187 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dn2cb" (UID: "92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10") : secret "insights-runtime-extractor-tls" not found Jul 10 19:01:04.960326 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:04.960168 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-rbac-proxy-cm podName:92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:08.960156567 +0000 UTC m=+50.496498978 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dn2cb" (UID: "92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10") : configmap "kube-rbac-proxy" not found Jul 10 19:01:08.993370 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:08.993332 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:08.993801 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:08.993380 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:08.993801 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:08.993444 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 19:01:08.993801 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:08.993490 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-rbac-proxy-cm podName:92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:16.993476091 +0000 UTC m=+58.529818500 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dn2cb" (UID: "92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10") : configmap "kube-rbac-proxy" not found Jul 10 19:01:08.993801 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:08.993498 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 19:01:08.993801 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:08.993572 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls podName:92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:16.993545332 +0000 UTC m=+58.529887741 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dn2cb" (UID: "92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10") : secret "insights-runtime-extractor-tls" not found Jul 10 19:01:09.194670 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:09.194637 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:01:09.194670 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:09.194669 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls\") pod \"dns-default-dptsw\" (UID: \"50e43ef8-5c11-457b-b37c-ca6711ecd556\") " pod="openshift-dns/dns-default-dptsw" Jul 10 19:01:09.194849 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:09.194779 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 19:01:09.194849 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:09.194784 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 19:01:09.194849 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:09.194800 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644b746fdc-rjsj4: secret "image-registry-tls" not found Jul 10 19:01:09.194849 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:09.194829 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls podName:50e43ef8-5c11-457b-b37c-ca6711ecd556 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:25.194813997 +0000 UTC m=+66.731156408 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls") pod "dns-default-dptsw" (UID: "50e43ef8-5c11-457b-b37c-ca6711ecd556") : secret "dns-default-metrics-tls" not found Jul 10 19:01:09.194849 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:09.194849 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls podName:5501e15d-59e4-4a41-8cb0-77ea136593ca nodeName:}" failed. No retries permitted until 2026-07-10 19:01:25.194835393 +0000 UTC m=+66.731177804 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls") pod "image-registry-644b746fdc-rjsj4" (UID: "5501e15d-59e4-4a41-8cb0-77ea136593ca") : secret "image-registry-tls" not found Jul 10 19:01:12.925847 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:12.925812 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-m7vls\" (UID: \"3af4a62b-153c-4bc5-82f6-722a7f1e5259\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" Jul 10 19:01:12.925847 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:12.925852 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:01:12.926364 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:12.925964 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle podName:97aba93c-11ba-49b1-8d28-3e26078983b2 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:28.925950845 +0000 UTC m=+70.462293256 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle") pod "router-default-68c78766fd-2z8xb" (UID: "97aba93c-11ba-49b1-8d28-3e26078983b2") : configmap references non-existent config key: service-ca.crt Jul 10 19:01:12.926364 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:12.925984 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:01:12.926364 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:12.925961 2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 19:01:12.926364 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:12.926079 2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 19:01:12.926364 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:12.926121 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs podName:97aba93c-11ba-49b1-8d28-3e26078983b2 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:28.926095794 +0000 UTC m=+70.462438204 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs") pod "router-default-68c78766fd-2z8xb" (UID: "97aba93c-11ba-49b1-8d28-3e26078983b2") : secret "router-metrics-certs-default" not found Jul 10 19:01:12.926364 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:12.926132 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls podName:3af4a62b-153c-4bc5-82f6-722a7f1e5259 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:28.926126719 +0000 UTC m=+70.462469129 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-m7vls" (UID: "3af4a62b-153c-4bc5-82f6-722a7f1e5259") : secret "cluster-monitoring-operator-tls" not found Jul 10 19:01:17.058770 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:17.058743 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:17.059217 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:17.058785 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:17.059217 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:17.058899 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 19:01:17.059217 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:17.058960 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls podName:92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:33.05894479 +0000 UTC m=+74.595287200 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dn2cb" (UID: "92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10") : secret "insights-runtime-extractor-tls" not found Jul 10 19:01:17.059345 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:17.059327 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:19.266232 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:19.266203 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kgpdv" Jul 10 19:01:19.315015 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:19.314972 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" podStartSLOduration=19.531980727 podStartE2EDuration="22.314957916s" totalImagesPullingTime="2.782977189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:00:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:01:01.317496423 +0000 UTC m=+42.853838856" watchObservedRunningTime="2026-07-10 19:01:19.314957916 +0000 UTC m=+60.851300347" Jul 10 19:01:24.821084 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:24.821040 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs\") pod \"network-metrics-daemon-9p2rq\" (UID: \"f911a124-c338-40f4-983f-c61a0d899d93\") " pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:01:24.821643 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:24.821094 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret\") pod \"global-pull-secret-syncer-4cdrm\" (UID: \"a7f690e5-5f82-4573-bc6e-a7ed083f6fbc\") " pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:01:24.824236 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:24.824213 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 19:01:24.824311 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:24.824240 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 19:01:24.831727 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:24.831708 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 19:01:24.831797 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:24.831777 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs podName:f911a124-c338-40f4-983f-c61a0d899d93 nodeName:}" failed. No retries permitted until 2026-07-10 19:02:28.831755931 +0000 UTC m=+130.368098346 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs") pod "network-metrics-daemon-9p2rq" (UID: "f911a124-c338-40f4-983f-c61a0d899d93") : secret "metrics-daemon-secret" not found Jul 10 19:01:24.833724 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:24.833703 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a7f690e5-5f82-4573-bc6e-a7ed083f6fbc-original-pull-secret\") pod \"global-pull-secret-syncer-4cdrm\" (UID: \"a7f690e5-5f82-4573-bc6e-a7ed083f6fbc\") " pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:01:24.921610 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:24.921573 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8qhq\" (UniqueName: \"kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq\") pod \"network-check-target-vjklm\" (UID: \"182da83f-63cb-4013-9868-4e95e297c58a\") " pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:01:24.924643 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:24.924623 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 19:01:24.934770 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:24.934747 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 19:01:24.936862 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:24.936848 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4cdrm" Jul 10 19:01:24.944777 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:24.944754 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8qhq\" (UniqueName: \"kubernetes.io/projected/182da83f-63cb-4013-9868-4e95e297c58a-kube-api-access-b8qhq\") pod \"network-check-target-vjklm\" (UID: \"182da83f-63cb-4013-9868-4e95e297c58a\") " pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:01:25.058297 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:25.058264 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4cdrm"] Jul 10 19:01:25.061246 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:25.061218 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7f690e5_5f82_4573_bc6e_a7ed083f6fbc.slice/crio-911d42acdfee3ae73b5c9eae8335e0e8ba8bae061eb5d578ff1d9e8908e3a39e WatchSource:0}: Error finding container 911d42acdfee3ae73b5c9eae8335e0e8ba8bae061eb5d578ff1d9e8908e3a39e: Status 404 returned error can't find the container with id 911d42acdfee3ae73b5c9eae8335e0e8ba8bae061eb5d578ff1d9e8908e3a39e Jul 10 19:01:25.225072 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:25.224995 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:01:25.225072 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:25.225033 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls\") pod \"dns-default-dptsw\" (UID: \"50e43ef8-5c11-457b-b37c-ca6711ecd556\") " pod="openshift-dns/dns-default-dptsw" Jul 10 19:01:25.226584 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:25.226567 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sqhv9\"" Jul 10 19:01:25.227331 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:25.227308 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50e43ef8-5c11-457b-b37c-ca6711ecd556-metrics-tls\") pod \"dns-default-dptsw\" (UID: \"50e43ef8-5c11-457b-b37c-ca6711ecd556\") " pod="openshift-dns/dns-default-dptsw" Jul 10 19:01:25.227519 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:25.227501 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls\") pod \"image-registry-644b746fdc-rjsj4\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:01:25.233912 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:25.233889 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:01:25.345551 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:25.345521 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4cdrm" event={"ID":"a7f690e5-5f82-4573-bc6e-a7ed083f6fbc","Type":"ContainerStarted","Data":"911d42acdfee3ae73b5c9eae8335e0e8ba8bae061eb5d578ff1d9e8908e3a39e"} Jul 10 19:01:25.350506 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:25.350479 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vjklm"] Jul 10 19:01:25.354281 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:25.354249 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod182da83f_63cb_4013_9868_4e95e297c58a.slice/crio-1e2fbb9d2451775a170f3d1c436601dd0ab92c7de77e0d5f2ec2efe25249cd40 WatchSource:0}: Error finding container 1e2fbb9d2451775a170f3d1c436601dd0ab92c7de77e0d5f2ec2efe25249cd40: Status 404 returned error can't find the container with id 1e2fbb9d2451775a170f3d1c436601dd0ab92c7de77e0d5f2ec2efe25249cd40 Jul 10 19:01:25.387619 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:25.387596 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-r62lv\"" Jul 10 19:01:25.395076 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:25.395058 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:01:25.399748 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:25.399727 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gmlfz\"" Jul 10 19:01:25.407235 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:25.407218 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dptsw" Jul 10 19:01:25.535474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:25.535445 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-644b746fdc-rjsj4"] Jul 10 19:01:25.539881 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:25.539853 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5501e15d_59e4_4a41_8cb0_77ea136593ca.slice/crio-e6c5f2ded7a0c3c09da9f4ea910d0c0a2d127ce051dbfc2deb934aed11106447 WatchSource:0}: Error finding container e6c5f2ded7a0c3c09da9f4ea910d0c0a2d127ce051dbfc2deb934aed11106447: Status 404 returned error can't find the container with id e6c5f2ded7a0c3c09da9f4ea910d0c0a2d127ce051dbfc2deb934aed11106447 Jul 10 19:01:25.550464 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:25.550443 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dptsw"] Jul 10 19:01:25.552996 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:25.552976 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50e43ef8_5c11_457b_b37c_ca6711ecd556.slice/crio-9ce1ff57d8000b2b8c290e24bdbd8a9a16773850bdb75e274f35ee225cc7e562 WatchSource:0}: Error finding container 9ce1ff57d8000b2b8c290e24bdbd8a9a16773850bdb75e274f35ee225cc7e562: Status 404 returned error can't find the container with id 9ce1ff57d8000b2b8c290e24bdbd8a9a16773850bdb75e274f35ee225cc7e562 Jul 10 19:01:26.356076 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:26.356032 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" event={"ID":"5501e15d-59e4-4a41-8cb0-77ea136593ca","Type":"ContainerStarted","Data":"5f3b07163d968face513f365c34a3200e6c9a3a0045a20ce67ed6f937a478908"} Jul 10 19:01:26.356557 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:26.356154 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" event={"ID":"5501e15d-59e4-4a41-8cb0-77ea136593ca","Type":"ContainerStarted","Data":"e6c5f2ded7a0c3c09da9f4ea910d0c0a2d127ce051dbfc2deb934aed11106447"} Jul 10 19:01:26.356557 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:26.356398 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:01:26.358370 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:26.358318 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vjklm" event={"ID":"182da83f-63cb-4013-9868-4e95e297c58a","Type":"ContainerStarted","Data":"1e2fbb9d2451775a170f3d1c436601dd0ab92c7de77e0d5f2ec2efe25249cd40"} Jul 10 19:01:26.360483 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:26.360426 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dptsw" event={"ID":"50e43ef8-5c11-457b-b37c-ca6711ecd556","Type":"ContainerStarted","Data":"9ce1ff57d8000b2b8c290e24bdbd8a9a16773850bdb75e274f35ee225cc7e562"} Jul 10 19:01:26.388427 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:26.388217 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" podStartSLOduration=45.388198683 podStartE2EDuration="45.388198683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:00:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:01:26.386776309 +0000 UTC m=+67.923118742" watchObservedRunningTime="2026-07-10 19:01:26.388198683 +0000 UTC m=+67.924541116" Jul 10 19:01:28.957186 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:28.957150 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-m7vls\" (UID: \"3af4a62b-153c-4bc5-82f6-722a7f1e5259\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" Jul 10 19:01:28.957636 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:28.957203 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:01:28.957636 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:28.957267 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:01:28.958036 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:28.958007 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97aba93c-11ba-49b1-8d28-3e26078983b2-service-ca-bundle\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:01:28.960137 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:28.960094 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3af4a62b-153c-4bc5-82f6-722a7f1e5259-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-m7vls\" (UID: \"3af4a62b-153c-4bc5-82f6-722a7f1e5259\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" Jul 10 19:01:28.960231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:28.960214 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97aba93c-11ba-49b1-8d28-3e26078983b2-metrics-certs\") pod \"router-default-68c78766fd-2z8xb\" (UID: \"97aba93c-11ba-49b1-8d28-3e26078983b2\") " pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:01:29.158060 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:29.158029 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-hb7cz\"" Jul 10 19:01:29.164685 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:29.164655 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" Jul 10 19:01:29.165285 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:29.165264 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-mb4h5\"" Jul 10 19:01:29.173680 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:29.173655 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:01:30.423552 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:30.423525 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68c78766fd-2z8xb"] Jul 10 19:01:30.426945 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:30.426903 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97aba93c_11ba_49b1_8d28_3e26078983b2.slice/crio-2a5da0f540e615519dd6e2e26216ba57d9c3dcf0f02c8c5ba56bd4ebad157d3f WatchSource:0}: Error finding container 2a5da0f540e615519dd6e2e26216ba57d9c3dcf0f02c8c5ba56bd4ebad157d3f: Status 404 returned error can't find the container with id 2a5da0f540e615519dd6e2e26216ba57d9c3dcf0f02c8c5ba56bd4ebad157d3f Jul 10 19:01:30.437914 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:30.437853 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls"] Jul 10 19:01:30.444083 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:30.444050 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3af4a62b_153c_4bc5_82f6_722a7f1e5259.slice/crio-af101ad6baa5ac15dd9b24d6289135e376fea50fc5acda944ddf16fb9e30c45d WatchSource:0}: Error finding container af101ad6baa5ac15dd9b24d6289135e376fea50fc5acda944ddf16fb9e30c45d: Status 404 returned error can't find the container with id af101ad6baa5ac15dd9b24d6289135e376fea50fc5acda944ddf16fb9e30c45d Jul 10 19:01:31.379858 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:31.379825 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" event={"ID":"3af4a62b-153c-4bc5-82f6-722a7f1e5259","Type":"ContainerStarted","Data":"af101ad6baa5ac15dd9b24d6289135e376fea50fc5acda944ddf16fb9e30c45d"} Jul 10 19:01:31.381457 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:31.381421 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4cdrm" event={"ID":"a7f690e5-5f82-4573-bc6e-a7ed083f6fbc","Type":"ContainerStarted","Data":"ebfd0472f2dee7d75bfae7b9ad064cd4286bf26206f84df9b83fa5ca4381712c"} Jul 10 19:01:31.383116 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:31.383062 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vjklm" event={"ID":"182da83f-63cb-4013-9868-4e95e297c58a","Type":"ContainerStarted","Data":"511d4c6ce8fdc4b1ae1ea48536f9365216e0667990f069c5207fcb8bcc96fde8"} Jul 10 19:01:31.383255 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:31.383147 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:01:31.384748 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:31.384721 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68c78766fd-2z8xb" event={"ID":"97aba93c-11ba-49b1-8d28-3e26078983b2","Type":"ContainerStarted","Data":"b0dfd8dbd212c1b647f4edab7d6dbd5a847d2524b543b4b45dfd886518703350"} Jul 10 19:01:31.384937 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:31.384764 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68c78766fd-2z8xb" event={"ID":"97aba93c-11ba-49b1-8d28-3e26078983b2","Type":"ContainerStarted","Data":"2a5da0f540e615519dd6e2e26216ba57d9c3dcf0f02c8c5ba56bd4ebad157d3f"} Jul 10 19:01:31.386808 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:31.386784 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dptsw" event={"ID":"50e43ef8-5c11-457b-b37c-ca6711ecd556","Type":"ContainerStarted","Data":"164f791ede1bdb37c2fb124497ea36bbd909ea899e90a16034099439369ad021"} Jul 10 19:01:31.386923 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:31.386828 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dptsw" event={"ID":"50e43ef8-5c11-457b-b37c-ca6711ecd556","Type":"ContainerStarted","Data":"1fdb44b4bb001e7597e3e109fe0b80debf7efef2cd26e7797d089758e5513106"} Jul 10 19:01:31.386991 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:31.386921 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dptsw" Jul 10 19:01:31.418085 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:31.418043 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dptsw" podStartSLOduration=33.704522682 podStartE2EDuration="38.418030528s" totalImagesPullingTime="4.713507846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:00:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:01:31.417631024 +0000 UTC m=+72.953973457" watchObservedRunningTime="2026-07-10 19:01:31.418030528 +0000 UTC m=+72.954372960" Jul 10 19:01:31.418474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:31.418450 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4cdrm" podStartSLOduration=67.202925372 podStartE2EDuration="1m12.418442614s" totalImagesPullingTime="5.215517242s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:01:31.398851647 +0000 UTC m=+72.935194080" watchObservedRunningTime="2026-07-10 19:01:31.418442614 +0000 UTC m=+72.954785046" Jul 10 19:01:31.437324 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:31.437285 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-68c78766fd-2z8xb" podStartSLOduration=34.437277605 podStartE2EDuration="34.437277605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:00:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:01:31.436363173 +0000 UTC m=+72.972705610" watchObservedRunningTime="2026-07-10 19:01:31.437277605 +0000 UTC m=+72.973620015" Jul 10 19:01:31.453619 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:31.453557 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vjklm" podStartSLOduration=67.536415488 podStartE2EDuration="1m12.453542732s" totalImagesPullingTime="4.917127244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:01:31.452602801 +0000 UTC m=+72.988945234" watchObservedRunningTime="2026-07-10 19:01:31.453542732 +0000 UTC m=+72.989885165" Jul 10 19:01:32.174607 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:32.174585 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:01:32.177375 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:32.177352 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:01:32.392230 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:32.392192 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" event={"ID":"3af4a62b-153c-4bc5-82f6-722a7f1e5259","Type":"ContainerStarted","Data":"4d2f995103ddc306d7839e7d78f09a7d82548305e471debd8138b5d4322bb64e"} Jul 10 19:01:32.392606 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:32.392573 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:01:32.393567 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:32.393549 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-68c78766fd-2z8xb" Jul 10 19:01:32.429252 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:32.426742 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-m7vls" podStartSLOduration=33.705612904 podStartE2EDuration="35.426724016s" totalImagesPullingTime="1.721111112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:00:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:01:32.425947214 +0000 UTC m=+73.962289637" watchObservedRunningTime="2026-07-10 19:01:32.426724016 +0000 UTC m=+73.963066448" Jul 10 19:01:33.095181 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:33.095150 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:33.097361 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:33.097331 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dn2cb\" (UID: \"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10\") " pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:33.353139 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:33.353055 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f67rl\"" Jul 10 19:01:33.360374 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:33.360353 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dn2cb" Jul 10 19:01:33.486162 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:33.486035 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dn2cb"] Jul 10 19:01:33.488540 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:33.488515 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92b6ef1d_16c3_4c77_bfd3_882a3bd1cf10.slice/crio-2ed423164f1345ffaa8e645ff192fd6e92bc858488cf0d177e5c236b2c2c7e33 WatchSource:0}: Error finding container 2ed423164f1345ffaa8e645ff192fd6e92bc858488cf0d177e5c236b2c2c7e33: Status 404 returned error can't find the container with id 2ed423164f1345ffaa8e645ff192fd6e92bc858488cf0d177e5c236b2c2c7e33 Jul 10 19:01:34.401864 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.401597 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dn2cb" event={"ID":"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10","Type":"ContainerStarted","Data":"2ed423164f1345ffaa8e645ff192fd6e92bc858488cf0d177e5c236b2c2c7e33"} Jul 10 19:01:34.546905 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.546864 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sgckr"] Jul 10 19:01:34.557794 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.557687 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-gb5kc"] Jul 10 19:01:34.557921 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.557813 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sgckr" Jul 10 19:01:34.559747 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.559725 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-gb5kc" Jul 10 19:01:34.568805 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.568782 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 10 19:01:34.570034 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.570011 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 10 19:01:34.570979 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.570953 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 10 19:01:34.571113 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.571087 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-rd544\"" Jul 10 19:01:34.587712 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.587696 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-sp7v4\"" Jul 10 19:01:34.604601 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.604581 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-gb5kc"] Jul 10 19:01:34.607816 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.607778 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sgckr"] Jul 10 19:01:34.708336 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.708262 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zgvx\" (UniqueName: \"kubernetes.io/projected/fd1dcfc5-bc32-448e-b38f-ce278c9992ff-kube-api-access-9zgvx\") pod \"downloads-68c9cbb497-gb5kc\" (UID: \"fd1dcfc5-bc32-448e-b38f-ce278c9992ff\") " pod="openshift-console/downloads-68c9cbb497-gb5kc" Jul 10 19:01:34.708336 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.708303 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/da2cf891-31e8-41b8-8a58-e8fce1192bdc-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-sgckr\" (UID: \"da2cf891-31e8-41b8-8a58-e8fce1192bdc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sgckr" Jul 10 19:01:34.809494 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.809456 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zgvx\" (UniqueName: \"kubernetes.io/projected/fd1dcfc5-bc32-448e-b38f-ce278c9992ff-kube-api-access-9zgvx\") pod \"downloads-68c9cbb497-gb5kc\" (UID: \"fd1dcfc5-bc32-448e-b38f-ce278c9992ff\") " pod="openshift-console/downloads-68c9cbb497-gb5kc" Jul 10 19:01:34.809679 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.809513 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/da2cf891-31e8-41b8-8a58-e8fce1192bdc-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-sgckr\" (UID: \"da2cf891-31e8-41b8-8a58-e8fce1192bdc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sgckr" Jul 10 19:01:34.812374 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.812346 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/da2cf891-31e8-41b8-8a58-e8fce1192bdc-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-sgckr\" (UID: \"da2cf891-31e8-41b8-8a58-e8fce1192bdc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sgckr" Jul 10 19:01:34.822804 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.822780 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zgvx\" (UniqueName: \"kubernetes.io/projected/fd1dcfc5-bc32-448e-b38f-ce278c9992ff-kube-api-access-9zgvx\") pod \"downloads-68c9cbb497-gb5kc\" (UID: \"fd1dcfc5-bc32-448e-b38f-ce278c9992ff\") " pod="openshift-console/downloads-68c9cbb497-gb5kc" Jul 10 19:01:34.870525 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.870496 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sgckr" Jul 10 19:01:34.877043 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:34.877016 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-gb5kc" Jul 10 19:01:35.066892 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:35.066859 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sgckr"] Jul 10 19:01:35.081093 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:35.081064 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-gb5kc"] Jul 10 19:01:35.958558 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:35.958519 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c57df6cf8-jpsmh"] Jul 10 19:01:35.972359 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:35.972337 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:35.975876 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:35.975717 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 10 19:01:35.975876 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:35.975845 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c57df6cf8-jpsmh"] Jul 10 19:01:35.976076 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:35.975849 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-pmw2s\"" Jul 10 19:01:35.976076 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:35.976035 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 10 19:01:35.976213 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:35.976076 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 10 19:01:35.977024 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:35.976830 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 10 19:01:35.977024 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:35.976846 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 10 19:01:36.120892 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.120861 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsxbc\" (UniqueName: \"kubernetes.io/projected/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-kube-api-access-zsxbc\") pod \"console-7c57df6cf8-jpsmh\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.121053 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.120897 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-console-serving-cert\") pod \"console-7c57df6cf8-jpsmh\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.121053 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.120935 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-console-config\") pod \"console-7c57df6cf8-jpsmh\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.121053 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.121002 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-console-oauth-config\") pod \"console-7c57df6cf8-jpsmh\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.121053 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.121043 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-oauth-serving-cert\") pod \"console-7c57df6cf8-jpsmh\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.121231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.121081 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-service-ca\") pod \"console-7c57df6cf8-jpsmh\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.221825 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.221759 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsxbc\" (UniqueName: \"kubernetes.io/projected/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-kube-api-access-zsxbc\") pod \"console-7c57df6cf8-jpsmh\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.221825 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.221791 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-console-serving-cert\") pod \"console-7c57df6cf8-jpsmh\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.221825 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.221820 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-console-config\") pod \"console-7c57df6cf8-jpsmh\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.222069 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.221838 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-console-oauth-config\") pod \"console-7c57df6cf8-jpsmh\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.222069 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.221857 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-oauth-serving-cert\") pod \"console-7c57df6cf8-jpsmh\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.222069 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.222039 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-service-ca\") pod \"console-7c57df6cf8-jpsmh\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.222658 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.222631 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-console-config\") pod \"console-7c57df6cf8-jpsmh\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.222781 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.222631 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-oauth-serving-cert\") pod \"console-7c57df6cf8-jpsmh\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.222781 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.222676 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-service-ca\") pod \"console-7c57df6cf8-jpsmh\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.224259 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.224231 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-console-oauth-config\") pod \"console-7c57df6cf8-jpsmh\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.224363 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.224343 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-console-serving-cert\") pod \"console-7c57df6cf8-jpsmh\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.230696 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.230672 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsxbc\" (UniqueName: \"kubernetes.io/projected/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-kube-api-access-zsxbc\") pod \"console-7c57df6cf8-jpsmh\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.258332 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:36.258309 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda2cf891_31e8_41b8_8a58_e8fce1192bdc.slice/crio-693b75a535329eb6ecf22d56996b9e23d3e7b8cc4e72d030f5bbbe8081a5a123 WatchSource:0}: Error finding container 693b75a535329eb6ecf22d56996b9e23d3e7b8cc4e72d030f5bbbe8081a5a123: Status 404 returned error can't find the container with id 693b75a535329eb6ecf22d56996b9e23d3e7b8cc4e72d030f5bbbe8081a5a123 Jul 10 19:01:36.285254 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.285186 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:36.293495 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:36.293470 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd1dcfc5_bc32_448e_b38f_ce278c9992ff.slice/crio-031b75af362ed89e1933299f5cd5b1b8ee20ec75a0a0d3b7c06f9c54c16ce9c5 WatchSource:0}: Error finding container 031b75af362ed89e1933299f5cd5b1b8ee20ec75a0a0d3b7c06f9c54c16ce9c5: Status 404 returned error can't find the container with id 031b75af362ed89e1933299f5cd5b1b8ee20ec75a0a0d3b7c06f9c54c16ce9c5 Jul 10 19:01:36.411039 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.411001 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-gb5kc" event={"ID":"fd1dcfc5-bc32-448e-b38f-ce278c9992ff","Type":"ContainerStarted","Data":"031b75af362ed89e1933299f5cd5b1b8ee20ec75a0a0d3b7c06f9c54c16ce9c5"} Jul 10 19:01:36.412210 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:36.412181 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sgckr" event={"ID":"da2cf891-31e8-41b8-8a58-e8fce1192bdc","Type":"ContainerStarted","Data":"693b75a535329eb6ecf22d56996b9e23d3e7b8cc4e72d030f5bbbe8081a5a123"} Jul 10 19:01:37.323576 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:37.323466 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c57df6cf8-jpsmh"] Jul 10 19:01:37.492380 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:37.492050 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd20bbb3_4733_4ea4_b5e8_d7fce01202b7.slice/crio-e0036b7b1eaa8c29d9cfe96e11e297c2a5d2fed381ff42a47452ae1bbbb1b278 WatchSource:0}: Error finding container e0036b7b1eaa8c29d9cfe96e11e297c2a5d2fed381ff42a47452ae1bbbb1b278: Status 404 returned error can't find the container with id e0036b7b1eaa8c29d9cfe96e11e297c2a5d2fed381ff42a47452ae1bbbb1b278 Jul 10 19:01:38.421420 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:38.421388 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dn2cb" event={"ID":"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10","Type":"ContainerStarted","Data":"720ba2e7982ee29e9a0aa6c6a8a06b203180903fde60fc81b5be47a34af7d9e5"} Jul 10 19:01:38.421784 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:38.421445 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dn2cb" event={"ID":"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10","Type":"ContainerStarted","Data":"4908a175f1c1a0e772da34c82903aaa35c743117c6d3e2869dc35ee398bf2664"} Jul 10 19:01:38.423199 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:38.423165 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sgckr" event={"ID":"da2cf891-31e8-41b8-8a58-e8fce1192bdc","Type":"ContainerStarted","Data":"c73449277511aa98ad6faaf6a6f08687081213448f32fa9a3121a9306aeefc67"} Jul 10 19:01:38.423412 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:38.423385 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sgckr" Jul 10 19:01:38.424550 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:38.424523 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c57df6cf8-jpsmh" event={"ID":"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7","Type":"ContainerStarted","Data":"e0036b7b1eaa8c29d9cfe96e11e297c2a5d2fed381ff42a47452ae1bbbb1b278"} Jul 10 19:01:38.428875 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:38.428850 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sgckr" Jul 10 19:01:38.446023 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:38.445973 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sgckr" podStartSLOduration=3.173623055 podStartE2EDuration="4.445957299s" totalImagesPullingTime="1.272334244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:01:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:01:38.4444388 +0000 UTC m=+79.980781234" watchObservedRunningTime="2026-07-10 19:01:38.445957299 +0000 UTC m=+79.982299731" Jul 10 19:01:38.798061 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:38.798028 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-ng62p"] Jul 10 19:01:38.800256 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:38.800236 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-ng62p" Jul 10 19:01:38.804574 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:38.804553 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-gng6l\"" Jul 10 19:01:38.804740 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:38.804715 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 19:01:38.804853 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:38.804772 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 10 19:01:38.805040 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:38.805023 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 10 19:01:38.812633 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:38.812609 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-ng62p"] Jul 10 19:01:38.942988 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:38.942941 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz2r4\" (UniqueName: \"kubernetes.io/projected/489ea98d-c7b4-4517-9c64-251ead911737-kube-api-access-jz2r4\") pod \"prometheus-operator-64785b6675-ng62p\" (UID: \"489ea98d-c7b4-4517-9c64-251ead911737\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ng62p" Jul 10 19:01:38.943177 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:38.943004 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/489ea98d-c7b4-4517-9c64-251ead911737-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-ng62p\" (UID: \"489ea98d-c7b4-4517-9c64-251ead911737\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ng62p" Jul 10 19:01:38.943177 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:38.943053 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/489ea98d-c7b4-4517-9c64-251ead911737-metrics-client-ca\") pod \"prometheus-operator-64785b6675-ng62p\" (UID: \"489ea98d-c7b4-4517-9c64-251ead911737\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ng62p" Jul 10 19:01:38.943177 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:38.943093 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/489ea98d-c7b4-4517-9c64-251ead911737-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-ng62p\" (UID: \"489ea98d-c7b4-4517-9c64-251ead911737\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ng62p" Jul 10 19:01:39.043929 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:39.043894 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/489ea98d-c7b4-4517-9c64-251ead911737-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-ng62p\" (UID: \"489ea98d-c7b4-4517-9c64-251ead911737\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ng62p" Jul 10 19:01:39.044080 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:39.043971 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz2r4\" (UniqueName: \"kubernetes.io/projected/489ea98d-c7b4-4517-9c64-251ead911737-kube-api-access-jz2r4\") pod \"prometheus-operator-64785b6675-ng62p\" (UID: \"489ea98d-c7b4-4517-9c64-251ead911737\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ng62p" Jul 10 19:01:39.044080 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:39.044023 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/489ea98d-c7b4-4517-9c64-251ead911737-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-ng62p\" (UID: \"489ea98d-c7b4-4517-9c64-251ead911737\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ng62p" Jul 10 19:01:39.044080 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:39.044074 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/489ea98d-c7b4-4517-9c64-251ead911737-metrics-client-ca\") pod \"prometheus-operator-64785b6675-ng62p\" (UID: \"489ea98d-c7b4-4517-9c64-251ead911737\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ng62p" Jul 10 19:01:39.044875 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:39.044847 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/489ea98d-c7b4-4517-9c64-251ead911737-metrics-client-ca\") pod \"prometheus-operator-64785b6675-ng62p\" (UID: \"489ea98d-c7b4-4517-9c64-251ead911737\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ng62p" Jul 10 19:01:39.047943 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:39.047896 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/489ea98d-c7b4-4517-9c64-251ead911737-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-ng62p\" (UID: \"489ea98d-c7b4-4517-9c64-251ead911737\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ng62p" Jul 10 19:01:39.049592 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:39.049530 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/489ea98d-c7b4-4517-9c64-251ead911737-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-ng62p\" (UID: \"489ea98d-c7b4-4517-9c64-251ead911737\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ng62p" Jul 10 19:01:39.055936 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:39.055908 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz2r4\" (UniqueName: \"kubernetes.io/projected/489ea98d-c7b4-4517-9c64-251ead911737-kube-api-access-jz2r4\") pod \"prometheus-operator-64785b6675-ng62p\" (UID: \"489ea98d-c7b4-4517-9c64-251ead911737\") " pod="openshift-monitoring/prometheus-operator-64785b6675-ng62p" Jul 10 19:01:39.111393 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:39.111362 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-ng62p" Jul 10 19:01:39.276586 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:39.276475 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-ng62p"] Jul 10 19:01:39.281265 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:39.281235 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod489ea98d_c7b4_4517_9c64_251ead911737.slice/crio-b3a2a0f9168def060ddf85d911ac83c1235b8516e3ef58e4aff7283af9992855 WatchSource:0}: Error finding container b3a2a0f9168def060ddf85d911ac83c1235b8516e3ef58e4aff7283af9992855: Status 404 returned error can't find the container with id b3a2a0f9168def060ddf85d911ac83c1235b8516e3ef58e4aff7283af9992855 Jul 10 19:01:39.432187 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:39.432088 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-ng62p" event={"ID":"489ea98d-c7b4-4517-9c64-251ead911737","Type":"ContainerStarted","Data":"b3a2a0f9168def060ddf85d911ac83c1235b8516e3ef58e4aff7283af9992855"} Jul 10 19:01:41.395260 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:41.394945 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dptsw" Jul 10 19:01:43.449049 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:43.449012 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dn2cb" event={"ID":"92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10","Type":"ContainerStarted","Data":"89aef7f9e71089589e96609e1a4385971b9101e6ed195d55bfa170d6758f869f"} Jul 10 19:01:43.450894 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:43.450863 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c57df6cf8-jpsmh" event={"ID":"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7","Type":"ContainerStarted","Data":"53e719f201e7e6427bf4e21eca920ab99aad21571d49ef1fcf1bc5ad442e6e1a"} Jul 10 19:01:43.452922 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:43.452889 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-ng62p" event={"ID":"489ea98d-c7b4-4517-9c64-251ead911737","Type":"ContainerStarted","Data":"3565399975ac019af80df6993148073fe85fce2408c3fda3142aab05b54dd318"} Jul 10 19:01:43.453021 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:43.452935 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-ng62p" event={"ID":"489ea98d-c7b4-4517-9c64-251ead911737","Type":"ContainerStarted","Data":"abdb71603d2de98beb31f9608d18e1d4344c117e0355df6c4faad4bfc8a0dcf1"} Jul 10 19:01:43.469875 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:43.469835 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dn2cb" podStartSLOduration=33.694146111 podStartE2EDuration="42.469821248s" totalImagesPullingTime="8.775675137s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:01:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:01:43.469174384 +0000 UTC m=+85.005516816" watchObservedRunningTime="2026-07-10 19:01:43.469821248 +0000 UTC m=+85.006163679" Jul 10 19:01:43.496148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:43.496088 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c57df6cf8-jpsmh" podStartSLOduration=3.598280962 podStartE2EDuration="8.496069852s" totalImagesPullingTime="4.89778889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:01:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:01:43.496014681 +0000 UTC m=+85.032357114" watchObservedRunningTime="2026-07-10 19:01:43.496069852 +0000 UTC m=+85.032412287" Jul 10 19:01:43.521610 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:43.521565 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-ng62p" podStartSLOduration=2.38875703 podStartE2EDuration="5.5215516s" totalImagesPullingTime="3.13279457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:01:43.520604222 +0000 UTC m=+85.056946654" watchObservedRunningTime="2026-07-10 19:01:43.5215516 +0000 UTC m=+85.057894009" Jul 10 19:01:44.342086 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.342056 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b78bb8dd5-xjcjp"] Jul 10 19:01:44.344488 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.344463 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.353064 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.353042 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 10 19:01:44.359290 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.359253 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b78bb8dd5-xjcjp"] Jul 10 19:01:44.392624 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.392594 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwktj\" (UniqueName: \"kubernetes.io/projected/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-kube-api-access-vwktj\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.392750 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.392640 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-service-ca\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.392750 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.392725 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-trusted-ca-bundle\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.392862 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.392757 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-oauth-serving-cert\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.392862 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.392807 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-console-serving-cert\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.392966 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.392866 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-console-oauth-config\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.392966 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.392921 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-console-config\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.493318 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.493283 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-trusted-ca-bundle\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.493318 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.493325 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-oauth-serving-cert\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.493771 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.493417 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-console-serving-cert\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.493771 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.493520 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-console-oauth-config\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.493771 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.493635 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-console-config\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.493771 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.493684 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwktj\" (UniqueName: \"kubernetes.io/projected/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-kube-api-access-vwktj\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.493771 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.493716 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-service-ca\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.494601 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.494287 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-trusted-ca-bundle\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.494601 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.494302 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-oauth-serving-cert\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.494601 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.494458 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-service-ca\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.495115 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.495072 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-console-config\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.496685 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.496659 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-console-oauth-config\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.496685 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.496681 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-console-serving-cert\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.503550 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.503530 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwktj\" (UniqueName: \"kubernetes.io/projected/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-kube-api-access-vwktj\") pod \"console-6b78bb8dd5-xjcjp\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.655925 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.655840 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:44.802829 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:44.802805 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b78bb8dd5-xjcjp"] Jul 10 19:01:44.805648 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:44.805621 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode49ee3e6_1da9_49bc_8fc4_5dfcd9250982.slice/crio-467c94a865b2d98beab8ee2289acfcfca0518dbb6e4610e080352fb3f42b7543 WatchSource:0}: Error finding container 467c94a865b2d98beab8ee2289acfcfca0518dbb6e4610e080352fb3f42b7543: Status 404 returned error can't find the container with id 467c94a865b2d98beab8ee2289acfcfca0518dbb6e4610e080352fb3f42b7543 Jul 10 19:01:45.171252 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.171215 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-khf86"] Jul 10 19:01:45.174323 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.174301 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" Jul 10 19:01:45.178387 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.178368 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-8tgxm\"" Jul 10 19:01:45.178508 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.178373 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 10 19:01:45.178608 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.178588 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 10 19:01:45.192280 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.192261 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-khf86"] Jul 10 19:01:45.201356 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.201317 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f7d20e2-9db3-4e12-af8d-2b179b92e50b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-khf86\" (UID: \"7f7d20e2-9db3-4e12-af8d-2b179b92e50b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" Jul 10 19:01:45.201472 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.201370 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f7d20e2-9db3-4e12-af8d-2b179b92e50b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-khf86\" (UID: \"7f7d20e2-9db3-4e12-af8d-2b179b92e50b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" Jul 10 19:01:45.201472 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.201436 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v8cz\" (UniqueName: \"kubernetes.io/projected/7f7d20e2-9db3-4e12-af8d-2b179b92e50b-kube-api-access-7v8cz\") pod \"openshift-state-metrics-84bd68b896-khf86\" (UID: \"7f7d20e2-9db3-4e12-af8d-2b179b92e50b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" Jul 10 19:01:45.201574 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.201486 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f7d20e2-9db3-4e12-af8d-2b179b92e50b-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-khf86\" (UID: \"7f7d20e2-9db3-4e12-af8d-2b179b92e50b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" Jul 10 19:01:45.212828 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.212802 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-79mdq"] Jul 10 19:01:45.215284 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.215263 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.218524 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.218309 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 19:01:45.218615 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.218577 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m7zfp\"" Jul 10 19:01:45.218720 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.218701 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 19:01:45.218848 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.218832 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 19:01:45.223173 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.223152 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-6gl5d"] Jul 10 19:01:45.225466 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.225442 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.228671 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.228652 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 10 19:01:45.229494 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.229468 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 10 19:01:45.230402 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.230377 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 10 19:01:45.231196 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.231177 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-v9cqs\"" Jul 10 19:01:45.244135 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.244110 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-6gl5d"] Jul 10 19:01:45.302680 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.302652 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3015d736-65f8-4b89-ad75-1f92c862b267-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-6gl5d\" (UID: \"3015d736-65f8-4b89-ad75-1f92c862b267\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.302811 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.302686 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-node-exporter-textfile\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.302811 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.302702 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3015d736-65f8-4b89-ad75-1f92c862b267-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-6gl5d\" (UID: \"3015d736-65f8-4b89-ad75-1f92c862b267\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.302811 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.302729 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-metrics-client-ca\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.302811 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.302758 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f7d20e2-9db3-4e12-af8d-2b179b92e50b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-khf86\" (UID: \"7f7d20e2-9db3-4e12-af8d-2b179b92e50b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" Jul 10 19:01:45.302811 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.302793 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f7d20e2-9db3-4e12-af8d-2b179b92e50b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-khf86\" (UID: \"7f7d20e2-9db3-4e12-af8d-2b179b92e50b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" Jul 10 19:01:45.303040 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.302820 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3015d736-65f8-4b89-ad75-1f92c862b267-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-6gl5d\" (UID: \"3015d736-65f8-4b89-ad75-1f92c862b267\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.303040 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.302856 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v8cz\" (UniqueName: \"kubernetes.io/projected/7f7d20e2-9db3-4e12-af8d-2b179b92e50b-kube-api-access-7v8cz\") pod \"openshift-state-metrics-84bd68b896-khf86\" (UID: \"7f7d20e2-9db3-4e12-af8d-2b179b92e50b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" Jul 10 19:01:45.303040 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.302882 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-sys\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.303040 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.302931 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v7vl\" (UniqueName: \"kubernetes.io/projected/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-kube-api-access-4v7vl\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.303040 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:45.302954 2571 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 10 19:01:45.303040 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.302984 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f7d20e2-9db3-4e12-af8d-2b179b92e50b-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-khf86\" (UID: \"7f7d20e2-9db3-4e12-af8d-2b179b92e50b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" Jul 10 19:01:45.303040 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.303016 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-node-exporter-accelerators-collector-config\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.303040 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:45.303033 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f7d20e2-9db3-4e12-af8d-2b179b92e50b-openshift-state-metrics-tls podName:7f7d20e2-9db3-4e12-af8d-2b179b92e50b nodeName:}" failed. No retries permitted until 2026-07-10 19:01:45.803011453 +0000 UTC m=+87.339353882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/7f7d20e2-9db3-4e12-af8d-2b179b92e50b-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-khf86" (UID: "7f7d20e2-9db3-4e12-af8d-2b179b92e50b") : secret "openshift-state-metrics-tls" not found Jul 10 19:01:45.303446 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.303162 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-node-exporter-wtmp\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.303446 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.303207 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-root\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.303446 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.303255 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.303446 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.303279 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2d9w\" (UniqueName: \"kubernetes.io/projected/3015d736-65f8-4b89-ad75-1f92c862b267-kube-api-access-f2d9w\") pod \"kube-state-metrics-785778ff74-6gl5d\" (UID: \"3015d736-65f8-4b89-ad75-1f92c862b267\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.303446 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.303331 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-node-exporter-tls\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.303446 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.303382 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3015d736-65f8-4b89-ad75-1f92c862b267-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-6gl5d\" (UID: \"3015d736-65f8-4b89-ad75-1f92c862b267\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.303446 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.303424 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3015d736-65f8-4b89-ad75-1f92c862b267-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-6gl5d\" (UID: \"3015d736-65f8-4b89-ad75-1f92c862b267\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.303806 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.303784 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f7d20e2-9db3-4e12-af8d-2b179b92e50b-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-khf86\" (UID: \"7f7d20e2-9db3-4e12-af8d-2b179b92e50b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" Jul 10 19:01:45.305699 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.305678 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f7d20e2-9db3-4e12-af8d-2b179b92e50b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-khf86\" (UID: \"7f7d20e2-9db3-4e12-af8d-2b179b92e50b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" Jul 10 19:01:45.315355 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.315312 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v8cz\" (UniqueName: \"kubernetes.io/projected/7f7d20e2-9db3-4e12-af8d-2b179b92e50b-kube-api-access-7v8cz\") pod \"openshift-state-metrics-84bd68b896-khf86\" (UID: \"7f7d20e2-9db3-4e12-af8d-2b179b92e50b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" Jul 10 19:01:45.400203 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.400166 2571 patch_prober.go:28] interesting pod/image-registry-644b746fdc-rjsj4 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 19:01:45.400322 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.400233 2571 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" podUID="5501e15d-59e4-4a41-8cb0-77ea136593ca" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 19:01:45.404364 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.404344 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-sys\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.404486 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.404370 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v7vl\" (UniqueName: \"kubernetes.io/projected/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-kube-api-access-4v7vl\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.404486 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.404408 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-node-exporter-accelerators-collector-config\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.404486 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.404438 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-node-exporter-wtmp\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.404486 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.404477 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-root\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.404486 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.404474 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-sys\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.404731 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.404531 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.404731 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.404555 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2d9w\" (UniqueName: \"kubernetes.io/projected/3015d736-65f8-4b89-ad75-1f92c862b267-kube-api-access-f2d9w\") pod \"kube-state-metrics-785778ff74-6gl5d\" (UID: \"3015d736-65f8-4b89-ad75-1f92c862b267\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.404731 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.404575 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-node-exporter-tls\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.404731 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.404590 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3015d736-65f8-4b89-ad75-1f92c862b267-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-6gl5d\" (UID: \"3015d736-65f8-4b89-ad75-1f92c862b267\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.404731 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.404611 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3015d736-65f8-4b89-ad75-1f92c862b267-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-6gl5d\" (UID: \"3015d736-65f8-4b89-ad75-1f92c862b267\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.404731 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.404624 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-node-exporter-wtmp\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.404731 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.404675 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3015d736-65f8-4b89-ad75-1f92c862b267-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-6gl5d\" (UID: \"3015d736-65f8-4b89-ad75-1f92c862b267\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.404731 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.404695 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-root\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.404731 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.404723 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-node-exporter-textfile\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.405080 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.404747 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3015d736-65f8-4b89-ad75-1f92c862b267-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-6gl5d\" (UID: \"3015d736-65f8-4b89-ad75-1f92c862b267\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.405080 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.404782 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-metrics-client-ca\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.405080 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.404826 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3015d736-65f8-4b89-ad75-1f92c862b267-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-6gl5d\" (UID: \"3015d736-65f8-4b89-ad75-1f92c862b267\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.405080 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.405037 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-node-exporter-textfile\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.405080 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.405044 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-node-exporter-accelerators-collector-config\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.405307 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.405081 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3015d736-65f8-4b89-ad75-1f92c862b267-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-6gl5d\" (UID: \"3015d736-65f8-4b89-ad75-1f92c862b267\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.405633 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.405561 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-metrics-client-ca\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.405633 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.405565 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3015d736-65f8-4b89-ad75-1f92c862b267-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-6gl5d\" (UID: \"3015d736-65f8-4b89-ad75-1f92c862b267\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.405793 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.405693 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3015d736-65f8-4b89-ad75-1f92c862b267-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-6gl5d\" (UID: \"3015d736-65f8-4b89-ad75-1f92c862b267\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.407354 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.407327 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.407659 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.407638 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-node-exporter-tls\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.407906 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.407884 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3015d736-65f8-4b89-ad75-1f92c862b267-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-6gl5d\" (UID: \"3015d736-65f8-4b89-ad75-1f92c862b267\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.408059 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.408040 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3015d736-65f8-4b89-ad75-1f92c862b267-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-6gl5d\" (UID: \"3015d736-65f8-4b89-ad75-1f92c862b267\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.418812 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.418792 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v7vl\" (UniqueName: \"kubernetes.io/projected/7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d-kube-api-access-4v7vl\") pod \"node-exporter-79mdq\" (UID: \"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d\") " pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.419978 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.419957 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2d9w\" (UniqueName: \"kubernetes.io/projected/3015d736-65f8-4b89-ad75-1f92c862b267-kube-api-access-f2d9w\") pod \"kube-state-metrics-785778ff74-6gl5d\" (UID: \"3015d736-65f8-4b89-ad75-1f92c862b267\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.463648 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.463589 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b78bb8dd5-xjcjp" event={"ID":"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982","Type":"ContainerStarted","Data":"58b2c3538dc56b7c98c54c47319293c7e91a923a7a45abddae27d5ea421ee347"} Jul 10 19:01:45.463648 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.463643 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b78bb8dd5-xjcjp" event={"ID":"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982","Type":"ContainerStarted","Data":"467c94a865b2d98beab8ee2289acfcfca0518dbb6e4610e080352fb3f42b7543"} Jul 10 19:01:45.487377 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.487333 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b78bb8dd5-xjcjp" podStartSLOduration=1.48731431 podStartE2EDuration="1.48731431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:01:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:01:45.486543144 +0000 UTC m=+87.022885578" watchObservedRunningTime="2026-07-10 19:01:45.48731431 +0000 UTC m=+87.023656742" Jul 10 19:01:45.525769 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.525739 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-79mdq" Jul 10 19:01:45.535467 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:45.535442 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d6f77fb_4ad4_47c3_a5bd_636b192c0b4d.slice/crio-4d2ab19ccec6d23dd01ae7009ede4fb5aa9aab1fc1d61ae01627672010ad83d2 WatchSource:0}: Error finding container 4d2ab19ccec6d23dd01ae7009ede4fb5aa9aab1fc1d61ae01627672010ad83d2: Status 404 returned error can't find the container with id 4d2ab19ccec6d23dd01ae7009ede4fb5aa9aab1fc1d61ae01627672010ad83d2 Jul 10 19:01:45.539641 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.539619 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" Jul 10 19:01:45.679433 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.679405 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-6gl5d"] Jul 10 19:01:45.682659 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:45.682622 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3015d736_65f8_4b89_ad75_1f92c862b267.slice/crio-5382036d50c9f56082ec1330afc5ef20e99c587d606a4eb1a4740a5d2f94caed WatchSource:0}: Error finding container 5382036d50c9f56082ec1330afc5ef20e99c587d606a4eb1a4740a5d2f94caed: Status 404 returned error can't find the container with id 5382036d50c9f56082ec1330afc5ef20e99c587d606a4eb1a4740a5d2f94caed Jul 10 19:01:45.809818 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.809789 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f7d20e2-9db3-4e12-af8d-2b179b92e50b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-khf86\" (UID: \"7f7d20e2-9db3-4e12-af8d-2b179b92e50b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" Jul 10 19:01:45.812132 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:45.812087 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f7d20e2-9db3-4e12-af8d-2b179b92e50b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-khf86\" (UID: \"7f7d20e2-9db3-4e12-af8d-2b179b92e50b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" Jul 10 19:01:46.084633 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.084553 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" Jul 10 19:01:46.258841 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.258759 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-khf86"] Jul 10 19:01:46.262581 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:46.262550 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f7d20e2_9db3_4e12_af8d_2b179b92e50b.slice/crio-4809e447196dad345f6ea4a43d825c58b5c82a198a559986b6f36bced06fe42a WatchSource:0}: Error finding container 4809e447196dad345f6ea4a43d825c58b5c82a198a559986b6f36bced06fe42a: Status 404 returned error can't find the container with id 4809e447196dad345f6ea4a43d825c58b5c82a198a559986b6f36bced06fe42a Jul 10 19:01:46.286279 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.286254 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:46.286407 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.286385 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:46.290225 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.290204 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 19:01:46.294354 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.294334 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:46.294595 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.294575 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.299563 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.299439 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 10 19:01:46.300527 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.300398 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 10 19:01:46.300527 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.300416 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-wcmlw\"" Jul 10 19:01:46.303436 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.303179 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 10 19:01:46.304672 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.303831 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 10 19:01:46.304672 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.303938 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 10 19:01:46.304672 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.303952 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 10 19:01:46.304672 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.304209 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 10 19:01:46.304672 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.304433 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 10 19:01:46.305459 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.304967 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 10 19:01:46.313432 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.313006 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/627e86ba-0df2-4638-a56c-d9f990f4ff36-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.313432 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.313045 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.313432 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.313076 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.313432 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.313140 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.313432 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.313166 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/627e86ba-0df2-4638-a56c-d9f990f4ff36-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.313432 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.313198 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.313432 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.313291 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.313837 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.313442 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/627e86ba-0df2-4638-a56c-d9f990f4ff36-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.313837 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.313474 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-config-volume\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.313837 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.313524 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-web-config\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.313837 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.313707 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/627e86ba-0df2-4638-a56c-d9f990f4ff36-tls-assets\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.313837 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.313774 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/627e86ba-0df2-4638-a56c-d9f990f4ff36-config-out\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.313837 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.313802 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhp94\" (UniqueName: \"kubernetes.io/projected/627e86ba-0df2-4638-a56c-d9f990f4ff36-kube-api-access-fhp94\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.322866 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.322783 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 19:01:46.414705 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.414244 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.414705 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.414295 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.414705 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.414385 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/627e86ba-0df2-4638-a56c-d9f990f4ff36-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.414705 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.414431 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-config-volume\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.414705 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.414467 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-web-config\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.414705 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.414512 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/627e86ba-0df2-4638-a56c-d9f990f4ff36-tls-assets\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.414705 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.414554 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/627e86ba-0df2-4638-a56c-d9f990f4ff36-config-out\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.414705 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.414587 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhp94\" (UniqueName: \"kubernetes.io/projected/627e86ba-0df2-4638-a56c-d9f990f4ff36-kube-api-access-fhp94\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.414705 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.414639 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/627e86ba-0df2-4638-a56c-d9f990f4ff36-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.414705 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.414679 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.415275 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.414724 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.415275 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.414755 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.415275 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.414782 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/627e86ba-0df2-4638-a56c-d9f990f4ff36-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.415275 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.414928 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/627e86ba-0df2-4638-a56c-d9f990f4ff36-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.415776 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.415548 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/627e86ba-0df2-4638-a56c-d9f990f4ff36-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.415860 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:01:46.415809 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/627e86ba-0df2-4638-a56c-d9f990f4ff36-alertmanager-trusted-ca-bundle podName:627e86ba-0df2-4638-a56c-d9f990f4ff36 nodeName:}" failed. No retries permitted until 2026-07-10 19:01:46.915787099 +0000 UTC m=+88.452129515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/627e86ba-0df2-4638-a56c-d9f990f4ff36-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "627e86ba-0df2-4638-a56c-d9f990f4ff36") : configmap references non-existent config key: ca-bundle.crt Jul 10 19:01:46.417644 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.417596 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-config-volume\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.418842 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.418795 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.419802 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.419589 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/627e86ba-0df2-4638-a56c-d9f990f4ff36-tls-assets\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.419802 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.419699 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.419802 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.419731 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.421807 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.421748 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.422111 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.421985 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.423817 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.422602 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/627e86ba-0df2-4638-a56c-d9f990f4ff36-config-out\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.423817 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.422643 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-web-config\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.425127 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.425064 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhp94\" (UniqueName: \"kubernetes.io/projected/627e86ba-0df2-4638-a56c-d9f990f4ff36-kube-api-access-fhp94\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.471281 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.471180 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" event={"ID":"3015d736-65f8-4b89-ad75-1f92c862b267","Type":"ContainerStarted","Data":"5382036d50c9f56082ec1330afc5ef20e99c587d606a4eb1a4740a5d2f94caed"} Jul 10 19:01:46.473463 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.473334 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" event={"ID":"7f7d20e2-9db3-4e12-af8d-2b179b92e50b","Type":"ContainerStarted","Data":"0afc5f40130a85d2fdb97f1bd520ff96075c1b87ad57d57c91763e3ab6a57958"} Jul 10 19:01:46.473463 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.473388 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" event={"ID":"7f7d20e2-9db3-4e12-af8d-2b179b92e50b","Type":"ContainerStarted","Data":"7f48664ceade12289e2bc86f1ab443a04fe10ce61c8889c1ca9fd734f1385b83"} Jul 10 19:01:46.473463 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.473415 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" event={"ID":"7f7d20e2-9db3-4e12-af8d-2b179b92e50b","Type":"ContainerStarted","Data":"4809e447196dad345f6ea4a43d825c58b5c82a198a559986b6f36bced06fe42a"} Jul 10 19:01:46.475546 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.475360 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-79mdq" event={"ID":"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d","Type":"ContainerStarted","Data":"4d2ab19ccec6d23dd01ae7009ede4fb5aa9aab1fc1d61ae01627672010ad83d2"} Jul 10 19:01:46.480723 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.480634 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:01:46.922137 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.922088 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/627e86ba-0df2-4638-a56c-d9f990f4ff36-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:46.922984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:46.922957 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/627e86ba-0df2-4638-a56c-d9f990f4ff36-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:47.210338 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:47.210030 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:01:47.369431 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:47.368941 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:01:47.482698 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:47.482665 2571 generic.go:356] "Generic (PLEG): container finished" podID="7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d" containerID="f53f895094f095ff517445078dd00f0d600e3b44df682a1933f8204492e8f4cf" exitCode=0 Jul 10 19:01:47.482861 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:47.482787 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-79mdq" event={"ID":"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d","Type":"ContainerDied","Data":"f53f895094f095ff517445078dd00f0d600e3b44df682a1933f8204492e8f4cf"} Jul 10 19:01:47.928968 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:47.928938 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 19:01:47.930570 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:47.930542 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod627e86ba_0df2_4638_a56c_d9f990f4ff36.slice/crio-7ba4a0f6e10d46e421058f102e21d86b30373d117e5507913734375677afcfc2 WatchSource:0}: Error finding container 7ba4a0f6e10d46e421058f102e21d86b30373d117e5507913734375677afcfc2: Status 404 returned error can't find the container with id 7ba4a0f6e10d46e421058f102e21d86b30373d117e5507913734375677afcfc2 Jul 10 19:01:48.490257 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:48.490226 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"627e86ba-0df2-4638-a56c-d9f990f4ff36","Type":"ContainerStarted","Data":"7ba4a0f6e10d46e421058f102e21d86b30373d117e5507913734375677afcfc2"} Jul 10 19:01:48.492092 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:48.492060 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" event={"ID":"7f7d20e2-9db3-4e12-af8d-2b179b92e50b","Type":"ContainerStarted","Data":"03ae4c8473cddb726a8e4cbf0a5b2a9e051c74fafa4e9dd62000b8f266b23526"} Jul 10 19:01:48.494209 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:48.494180 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-79mdq" event={"ID":"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d","Type":"ContainerStarted","Data":"9053fc9719b681765d531b82127a8a58c0852b84a671310da2562441361e7237"} Jul 10 19:01:48.494323 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:48.494231 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-79mdq" event={"ID":"7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d","Type":"ContainerStarted","Data":"627ca3d86c804361319778337049b71223d16fdced0d6ff148381ae080f596fa"} Jul 10 19:01:48.496156 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:48.496127 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" event={"ID":"3015d736-65f8-4b89-ad75-1f92c862b267","Type":"ContainerStarted","Data":"be8e316d12cf52a6d1af869b308e19ac6c95603ddda8e7d333435d388c109206"} Jul 10 19:01:48.496244 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:48.496181 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" event={"ID":"3015d736-65f8-4b89-ad75-1f92c862b267","Type":"ContainerStarted","Data":"f0ab7c58d98117a355026804587029cdb531366537bf2f5e971fb2dc1bc1384b"} Jul 10 19:01:48.496244 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:48.496210 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" event={"ID":"3015d736-65f8-4b89-ad75-1f92c862b267","Type":"ContainerStarted","Data":"dbb94477116a3ed9ae5e6c559b1ecdd62d1fd44fa5fce85f6f46d680dd90cc8a"} Jul 10 19:01:48.515682 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:48.515640 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-khf86" podStartSLOduration=2.174085323 podStartE2EDuration="3.515626413s" totalImagesPullingTime="1.34154109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:01:48.514018173 +0000 UTC m=+90.050360607" watchObservedRunningTime="2026-07-10 19:01:48.515626413 +0000 UTC m=+90.051968823" Jul 10 19:01:48.565526 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:48.565485 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-6gl5d" podStartSLOduration=1.501884628 podStartE2EDuration="3.565470064s" totalImagesPullingTime="2.063585436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:01:48.539486667 +0000 UTC m=+90.075829100" watchObservedRunningTime="2026-07-10 19:01:48.565470064 +0000 UTC m=+90.101812495" Jul 10 19:01:48.566009 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:48.565976 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-79mdq" podStartSLOduration=2.106277463 podStartE2EDuration="3.565961205s" totalImagesPullingTime="1.381260494s" totalInitContainerRuntime="78.423248ms" isStatefulPod=true podCreationTimestamp="2026-07-10 19:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:01:48.565052894 +0000 UTC m=+90.101395327" watchObservedRunningTime="2026-07-10 19:01:48.565961205 +0000 UTC m=+90.102303641" Jul 10 19:01:49.623545 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.623508 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-78b7bbd787-cstjx"] Jul 10 19:01:49.626821 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.626793 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.630911 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.630885 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 10 19:01:49.631032 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.630956 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 10 19:01:49.631032 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.630996 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 10 19:01:49.631270 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.631248 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-aaaori1a3ikiu\"" Jul 10 19:01:49.631343 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.631298 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-z9mqd\"" Jul 10 19:01:49.631382 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.631367 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 10 19:01:49.645807 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.645781 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a64a56f1-0781-4eca-bd77-bb38a9af297f-audit-log\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.645886 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.645820 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a64a56f1-0781-4eca-bd77-bb38a9af297f-metrics-server-audit-profiles\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.645886 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.645848 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a64a56f1-0781-4eca-bd77-bb38a9af297f-secret-metrics-server-tls\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.645986 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.645920 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a64a56f1-0781-4eca-bd77-bb38a9af297f-client-ca-bundle\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.645986 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.645972 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a64a56f1-0781-4eca-bd77-bb38a9af297f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.646133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.646080 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a64a56f1-0781-4eca-bd77-bb38a9af297f-secret-metrics-server-client-certs\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.646133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.646127 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd47m\" (UniqueName: \"kubernetes.io/projected/a64a56f1-0781-4eca-bd77-bb38a9af297f-kube-api-access-hd47m\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.647177 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.647153 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78b7bbd787-cstjx"] Jul 10 19:01:49.747599 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.747564 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a64a56f1-0781-4eca-bd77-bb38a9af297f-secret-metrics-server-client-certs\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.747599 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.747597 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd47m\" (UniqueName: \"kubernetes.io/projected/a64a56f1-0781-4eca-bd77-bb38a9af297f-kube-api-access-hd47m\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.747843 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.747640 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a64a56f1-0781-4eca-bd77-bb38a9af297f-audit-log\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.747843 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.747664 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a64a56f1-0781-4eca-bd77-bb38a9af297f-metrics-server-audit-profiles\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.747843 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.747690 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a64a56f1-0781-4eca-bd77-bb38a9af297f-secret-metrics-server-tls\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.747843 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.747725 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a64a56f1-0781-4eca-bd77-bb38a9af297f-client-ca-bundle\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.747843 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.747767 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a64a56f1-0781-4eca-bd77-bb38a9af297f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.748184 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.748155 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a64a56f1-0781-4eca-bd77-bb38a9af297f-audit-log\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.748632 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.748602 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a64a56f1-0781-4eca-bd77-bb38a9af297f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.748877 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.748853 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a64a56f1-0781-4eca-bd77-bb38a9af297f-metrics-server-audit-profiles\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.750582 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.750557 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a64a56f1-0781-4eca-bd77-bb38a9af297f-secret-metrics-server-client-certs\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.750689 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.750655 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a64a56f1-0781-4eca-bd77-bb38a9af297f-secret-metrics-server-tls\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.750814 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.750792 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a64a56f1-0781-4eca-bd77-bb38a9af297f-client-ca-bundle\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.757948 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.757903 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd47m\" (UniqueName: \"kubernetes.io/projected/a64a56f1-0781-4eca-bd77-bb38a9af297f-kube-api-access-hd47m\") pod \"metrics-server-78b7bbd787-cstjx\" (UID: \"a64a56f1-0781-4eca-bd77-bb38a9af297f\") " pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:49.937827 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:49.937745 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:01:50.460143 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.460095 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-64975d4fc5-mb6s6"] Jul 10 19:01:50.463719 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.463696 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.467068 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.466674 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 10 19:01:50.467068 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.466705 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 10 19:01:50.467068 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.466751 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 10 19:01:50.467068 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.466861 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 10 19:01:50.467068 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.466886 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-88ntq\"" Jul 10 19:01:50.467068 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.466931 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 10 19:01:50.473192 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.473173 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 10 19:01:50.480720 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.480697 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-64975d4fc5-mb6s6"] Jul 10 19:01:50.554850 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.554817 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/379b8e10-3447-4632-af6c-8f392b1f23d5-telemeter-client-tls\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.555010 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.554857 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/379b8e10-3447-4632-af6c-8f392b1f23d5-serving-certs-ca-bundle\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.555010 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.554922 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng7gd\" (UniqueName: \"kubernetes.io/projected/379b8e10-3447-4632-af6c-8f392b1f23d5-kube-api-access-ng7gd\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.555010 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.554971 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/379b8e10-3447-4632-af6c-8f392b1f23d5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.555010 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.555000 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/379b8e10-3447-4632-af6c-8f392b1f23d5-federate-client-tls\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.555249 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.555073 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/379b8e10-3447-4632-af6c-8f392b1f23d5-secret-telemeter-client\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.555249 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.555189 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/379b8e10-3447-4632-af6c-8f392b1f23d5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.555249 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.555233 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/379b8e10-3447-4632-af6c-8f392b1f23d5-metrics-client-ca\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.656226 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.656187 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/379b8e10-3447-4632-af6c-8f392b1f23d5-secret-telemeter-client\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.656663 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.656281 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/379b8e10-3447-4632-af6c-8f392b1f23d5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.656663 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.656323 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/379b8e10-3447-4632-af6c-8f392b1f23d5-metrics-client-ca\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.656663 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.656385 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/379b8e10-3447-4632-af6c-8f392b1f23d5-telemeter-client-tls\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.656663 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.656412 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/379b8e10-3447-4632-af6c-8f392b1f23d5-serving-certs-ca-bundle\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.656663 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.656439 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng7gd\" (UniqueName: \"kubernetes.io/projected/379b8e10-3447-4632-af6c-8f392b1f23d5-kube-api-access-ng7gd\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.656663 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.656473 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/379b8e10-3447-4632-af6c-8f392b1f23d5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.656663 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.656501 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/379b8e10-3447-4632-af6c-8f392b1f23d5-federate-client-tls\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.657390 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.657345 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/379b8e10-3447-4632-af6c-8f392b1f23d5-metrics-client-ca\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.658013 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.657939 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/379b8e10-3447-4632-af6c-8f392b1f23d5-serving-certs-ca-bundle\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.658855 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.658788 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/379b8e10-3447-4632-af6c-8f392b1f23d5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.659499 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.659467 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/379b8e10-3447-4632-af6c-8f392b1f23d5-federate-client-tls\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.659585 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.659532 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/379b8e10-3447-4632-af6c-8f392b1f23d5-secret-telemeter-client\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.659876 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.659849 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/379b8e10-3447-4632-af6c-8f392b1f23d5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.659964 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.659888 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/379b8e10-3447-4632-af6c-8f392b1f23d5-telemeter-client-tls\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.666271 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.666246 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng7gd\" (UniqueName: \"kubernetes.io/projected/379b8e10-3447-4632-af6c-8f392b1f23d5-kube-api-access-ng7gd\") pod \"telemeter-client-64975d4fc5-mb6s6\" (UID: \"379b8e10-3447-4632-af6c-8f392b1f23d5\") " pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:50.755416 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.755345 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b78bb8dd5-xjcjp"] Jul 10 19:01:50.774520 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:50.774491 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" Jul 10 19:01:51.499989 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.499952 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 19:01:51.505260 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.505236 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.508618 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.508591 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 10 19:01:51.508830 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.508813 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 10 19:01:51.509875 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.508985 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 10 19:01:51.509875 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.509675 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-m9wkx\"" Jul 10 19:01:51.510708 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.510685 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 10 19:01:51.510827 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.510807 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 10 19:01:51.512134 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.510918 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 10 19:01:51.512134 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.511047 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 10 19:01:51.512134 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.511148 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 10 19:01:51.512134 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.511279 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 10 19:01:51.512134 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.511368 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 10 19:01:51.512134 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.511530 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b13s5uq4qjc4e\"" Jul 10 19:01:51.512134 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.511603 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 10 19:01:51.512134 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.511818 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 10 19:01:51.512134 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.511967 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 10 19:01:51.513968 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.513951 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 10 19:01:51.525852 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.525829 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 19:01:51.566207 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.566171 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.566497 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.566271 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/15c278e5-f3a7-438a-8f98-d0acbadf2484-config-out\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.566497 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.566307 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.566497 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.566448 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.566497 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.566490 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.566717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.566518 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.566717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.566563 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.566717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.566605 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.566717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.566663 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnrl9\" (UniqueName: \"kubernetes.io/projected/15c278e5-f3a7-438a-8f98-d0acbadf2484-kube-api-access-nnrl9\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.566717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.566703 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-config\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.567039 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.566729 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.567039 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.566759 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.567039 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.566786 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/15c278e5-f3a7-438a-8f98-d0acbadf2484-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.567039 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.566808 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.567039 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.566848 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.567039 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.566874 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.567039 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.566900 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.567039 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.567025 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.567397 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.567070 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-web-config\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.567397 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.567132 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.668272 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.668229 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnrl9\" (UniqueName: \"kubernetes.io/projected/15c278e5-f3a7-438a-8f98-d0acbadf2484-kube-api-access-nnrl9\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.668726 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.668281 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-config\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.668726 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.668308 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.668726 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.668335 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.668726 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.668616 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/15c278e5-f3a7-438a-8f98-d0acbadf2484-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.668946 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.668783 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.669442 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.668821 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.669551 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.669476 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.669551 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.669510 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.669661 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.669548 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.669661 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.669577 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-web-config\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.669855 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.669830 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.669918 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.669892 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.670007 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.669987 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/15c278e5-f3a7-438a-8f98-d0acbadf2484-config-out\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.670068 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.670022 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.670152 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.670112 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.670152 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.670144 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.670261 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.670173 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.670261 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.670197 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.670261 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.670225 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.671455 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.671426 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.671549 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.671528 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.672036 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.672011 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.673635 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.673604 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.674033 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.674012 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-web-config\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.674435 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.674408 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.675093 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.675066 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.676253 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.676205 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.676827 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.676762 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.677653 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.677203 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/15c278e5-f3a7-438a-8f98-d0acbadf2484-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.677653 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.677278 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-config\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.677653 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.677526 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.677852 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.677665 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.677905 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.677844 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/15c278e5-f3a7-438a-8f98-d0acbadf2484-config-out\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.678123 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.678077 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.678213 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.678128 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.678277 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.678257 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.678333 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.678311 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.678472 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.678427 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.685078 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.685055 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnrl9\" (UniqueName: \"kubernetes.io/projected/15c278e5-f3a7-438a-8f98-d0acbadf2484-kube-api-access-nnrl9\") pod \"prometheus-k8s-0\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:51.820709 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:51.818753 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:01:54.656341 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:54.656307 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:01:57.332881 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:57.332857 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-64975d4fc5-mb6s6"] Jul 10 19:01:57.337136 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:57.337113 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-644b746fdc-rjsj4"] Jul 10 19:01:57.368153 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:57.368122 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod379b8e10_3447_4632_af6c_8f392b1f23d5.slice/crio-c156f5f957f17a68150f2bff3e1c0f962d5645ce50ee4679dc0e2f84dc88e597 WatchSource:0}: Error finding container c156f5f957f17a68150f2bff3e1c0f962d5645ce50ee4679dc0e2f84dc88e597: Status 404 returned error can't find the container with id c156f5f957f17a68150f2bff3e1c0f962d5645ce50ee4679dc0e2f84dc88e597 Jul 10 19:01:57.546583 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:57.546541 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-gb5kc" event={"ID":"fd1dcfc5-bc32-448e-b38f-ce278c9992ff","Type":"ContainerStarted","Data":"732245871c79f15b8b5743ddf350e02d072c3372f92432d25fd1f02f2d0aaedf"} Jul 10 19:01:57.547138 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:57.547082 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-gb5kc" Jul 10 19:01:57.549276 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:57.549246 2571 generic.go:356] "Generic (PLEG): container finished" podID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerID="44a17d61cad01ac43dd6752194bc8e0e258eea55292c5988c18800c9f5977847" exitCode=0 Jul 10 19:01:57.549411 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:57.549329 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"627e86ba-0df2-4638-a56c-d9f990f4ff36","Type":"ContainerDied","Data":"44a17d61cad01ac43dd6752194bc8e0e258eea55292c5988c18800c9f5977847"} Jul 10 19:01:57.551950 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:57.551807 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" event={"ID":"379b8e10-3447-4632-af6c-8f392b1f23d5","Type":"ContainerStarted","Data":"c156f5f957f17a68150f2bff3e1c0f962d5645ce50ee4679dc0e2f84dc88e597"} Jul 10 19:01:57.553793 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:57.553770 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78b7bbd787-cstjx"] Jul 10 19:01:57.555632 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:57.555603 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda64a56f1_0781_4eca_bd77_bb38a9af297f.slice/crio-5a90e3b9c3a8d56ad8b43088788f2d083531a799f450da94623ef5dfc8ad22af WatchSource:0}: Error finding container 5a90e3b9c3a8d56ad8b43088788f2d083531a799f450da94623ef5dfc8ad22af: Status 404 returned error can't find the container with id 5a90e3b9c3a8d56ad8b43088788f2d083531a799f450da94623ef5dfc8ad22af Jul 10 19:01:57.557155 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:57.557129 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-gb5kc" Jul 10 19:01:57.563148 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:57.563071 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 19:01:57.565655 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:57.565600 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-gb5kc" podStartSLOduration=2.730147883 podStartE2EDuration="23.565588907s" totalImagesPullingTime="20.835441024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:01:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:01:57.564088682 +0000 UTC m=+99.100431108" watchObservedRunningTime="2026-07-10 19:01:57.565588907 +0000 UTC m=+99.101931339" Jul 10 19:01:57.566069 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:01:57.566043 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c278e5_f3a7_438a_8f98_d0acbadf2484.slice/crio-ed8804f251bf34f9b8bcd94df0cb24902dedc364f5c72fac93029c801d11aa64 WatchSource:0}: Error finding container ed8804f251bf34f9b8bcd94df0cb24902dedc364f5c72fac93029c801d11aa64: Status 404 returned error can't find the container with id ed8804f251bf34f9b8bcd94df0cb24902dedc364f5c72fac93029c801d11aa64 Jul 10 19:01:58.560145 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:58.560044 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" event={"ID":"a64a56f1-0781-4eca-bd77-bb38a9af297f","Type":"ContainerStarted","Data":"5a90e3b9c3a8d56ad8b43088788f2d083531a799f450da94623ef5dfc8ad22af"} Jul 10 19:01:58.564309 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:58.562665 2571 generic.go:356] "Generic (PLEG): container finished" podID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerID="8d7cfd7d05ec94380d7e4a72ff312d4735f7895781bce64df7c695b755004e19" exitCode=0 Jul 10 19:01:58.564309 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:58.563976 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15c278e5-f3a7-438a-8f98-d0acbadf2484","Type":"ContainerDied","Data":"8d7cfd7d05ec94380d7e4a72ff312d4735f7895781bce64df7c695b755004e19"} Jul 10 19:01:58.564309 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:01:58.564038 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15c278e5-f3a7-438a-8f98-d0acbadf2484","Type":"ContainerStarted","Data":"ed8804f251bf34f9b8bcd94df0cb24902dedc364f5c72fac93029c801d11aa64"} Jul 10 19:02:01.477487 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:01.477346 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c57df6cf8-jpsmh"] Jul 10 19:02:02.395529 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:02.395496 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vjklm" Jul 10 19:02:02.593038 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:02.593000 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" event={"ID":"a64a56f1-0781-4eca-bd77-bb38a9af297f","Type":"ContainerStarted","Data":"e208a250818dbb59a60e0c2e1775c7e997271a66acb2c6823736f5a70a840888"} Jul 10 19:02:02.597913 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:02.597652 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"627e86ba-0df2-4638-a56c-d9f990f4ff36","Type":"ContainerStarted","Data":"dd8c352cd70202f955612aa8059fd5b2d8cdd449d5ed41d08f05e2b7bc020176"} Jul 10 19:02:02.597913 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:02.597735 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"627e86ba-0df2-4638-a56c-d9f990f4ff36","Type":"ContainerStarted","Data":"7b1f88ab126f1d8f3d2439937d0378e303ec11cbbf367ceb5b15818a875232aa"} Jul 10 19:02:02.597913 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:02.597809 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"627e86ba-0df2-4638-a56c-d9f990f4ff36","Type":"ContainerStarted","Data":"1c4e9e43a1b476d5b55f96997a042cf63e2023eb90694d2651f319da7ce291e9"} Jul 10 19:02:02.597913 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:02.597849 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"627e86ba-0df2-4638-a56c-d9f990f4ff36","Type":"ContainerStarted","Data":"941dc1333d8adb2ec9fe3696ded34f0d7e12879673aab5349d6a810b4690a655"} Jul 10 19:02:02.601533 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:02.601309 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" event={"ID":"379b8e10-3447-4632-af6c-8f392b1f23d5","Type":"ContainerStarted","Data":"6089b4b8259ee5980810e01e0cb4043d2360a35cd4f0ea34cd00eb5650e8e270"} Jul 10 19:02:02.601533 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:02.601378 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" event={"ID":"379b8e10-3447-4632-af6c-8f392b1f23d5","Type":"ContainerStarted","Data":"46b27e6dcd78a84661124a80661017b827a7feaf866aa7f3582c73fe30912939"} Jul 10 19:02:02.601533 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:02.601410 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" event={"ID":"379b8e10-3447-4632-af6c-8f392b1f23d5","Type":"ContainerStarted","Data":"7313bea616b660372dcd27ec462829b23021ef17331d8d8ba103eb15cafb6749"} Jul 10 19:02:02.652032 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:02.651937 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-64975d4fc5-mb6s6" podStartSLOduration=8.530122806 podStartE2EDuration="12.651916323s" totalImagesPullingTime="4.121793517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:01:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:02:02.649918739 +0000 UTC m=+104.186261174" watchObservedRunningTime="2026-07-10 19:02:02.651916323 +0000 UTC m=+104.188258959" Jul 10 19:02:02.652812 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:02.652769 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" podStartSLOduration=9.714553018 podStartE2EDuration="13.652752702s" totalImagesPullingTime="3.938199684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:01:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:02:02.617251582 +0000 UTC m=+104.153594016" watchObservedRunningTime="2026-07-10 19:02:02.652752702 +0000 UTC m=+104.189095135" Jul 10 19:02:03.360174 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.360140 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f4c68b74f-vprxv"] Jul 10 19:02:03.387475 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.386713 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f4c68b74f-vprxv"] Jul 10 19:02:03.387475 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.386833 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.516526 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.516483 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-service-ca\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.516526 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.516532 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-oauth-serving-cert\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.516792 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.516557 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgx5t\" (UniqueName: \"kubernetes.io/projected/99f22134-a60c-4ed5-9d46-8e0c15815d38-kube-api-access-hgx5t\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.516792 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.516590 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99f22134-a60c-4ed5-9d46-8e0c15815d38-console-oauth-config\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.516792 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.516673 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99f22134-a60c-4ed5-9d46-8e0c15815d38-console-serving-cert\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.516792 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.516715 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-trusted-ca-bundle\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.516986 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.516816 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-console-config\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.618893 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.618812 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-console-config\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.619431 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.618942 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-service-ca\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.619431 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.618965 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-oauth-serving-cert\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.619431 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.618985 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgx5t\" (UniqueName: \"kubernetes.io/projected/99f22134-a60c-4ed5-9d46-8e0c15815d38-kube-api-access-hgx5t\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.619431 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.619017 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99f22134-a60c-4ed5-9d46-8e0c15815d38-console-oauth-config\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.620081 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.619741 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-service-ca\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.620081 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.619776 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99f22134-a60c-4ed5-9d46-8e0c15815d38-console-serving-cert\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.620081 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.619813 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-trusted-ca-bundle\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.620081 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.619962 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-oauth-serving-cert\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.620081 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.619744 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-console-config\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.620642 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.620615 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-trusted-ca-bundle\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.622925 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.622902 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99f22134-a60c-4ed5-9d46-8e0c15815d38-console-oauth-config\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.627925 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.627880 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99f22134-a60c-4ed5-9d46-8e0c15815d38-console-serving-cert\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.628867 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.628824 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgx5t\" (UniqueName: \"kubernetes.io/projected/99f22134-a60c-4ed5-9d46-8e0c15815d38-kube-api-access-hgx5t\") pod \"console-f4c68b74f-vprxv\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.701765 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.701727 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:03.902434 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:03.902309 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f4c68b74f-vprxv"] Jul 10 19:02:04.615996 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:04.615957 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f4c68b74f-vprxv" event={"ID":"99f22134-a60c-4ed5-9d46-8e0c15815d38","Type":"ContainerStarted","Data":"67b3681b3b567d5e0d1f2ee3e058908a449d2dee870d26742ad44969d9cc5d09"} Jul 10 19:02:04.616280 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:04.616020 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f4c68b74f-vprxv" event={"ID":"99f22134-a60c-4ed5-9d46-8e0c15815d38","Type":"ContainerStarted","Data":"13800a0120c35d1fa2adf21be339ea87721f27d6286a4484b1a14e3daa211eb8"} Jul 10 19:02:04.620928 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:04.620892 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"627e86ba-0df2-4638-a56c-d9f990f4ff36","Type":"ContainerStarted","Data":"1b7fa350416c7620b651f2c524385d85c2d41a22b38fad750bf9ae979cef5b0f"} Jul 10 19:02:04.623878 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:04.623845 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15c278e5-f3a7-438a-8f98-d0acbadf2484","Type":"ContainerStarted","Data":"cd1696abe9f32e82f78097fa8f10577f2ec84ce55bcdae3813b4f853d7526f13"} Jul 10 19:02:04.623979 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:04.623914 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15c278e5-f3a7-438a-8f98-d0acbadf2484","Type":"ContainerStarted","Data":"89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88"} Jul 10 19:02:04.643068 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:04.643016 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f4c68b74f-vprxv" podStartSLOduration=1.643001134 podStartE2EDuration="1.643001134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:02:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:02:04.642484132 +0000 UTC m=+106.178826565" watchObservedRunningTime="2026-07-10 19:02:04.643001134 +0000 UTC m=+106.179343560" Jul 10 19:02:05.635231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:05.635195 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"627e86ba-0df2-4638-a56c-d9f990f4ff36","Type":"ContainerStarted","Data":"f993869d344244ed8ffe53536c3b28702130b0360c316fee9443277677f5493a"} Jul 10 19:02:05.676383 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:05.675492 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=5.133325788 podStartE2EDuration="19.675451818s" totalImagesPullingTime="14.524037971s" totalInitContainerRuntime="18.088059ms" isStatefulPod=false podCreationTimestamp="2026-07-10 19:01:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:02:05.674950764 +0000 UTC m=+107.211293218" watchObservedRunningTime="2026-07-10 19:02:05.675451818 +0000 UTC m=+107.211794252" Jul 10 19:02:07.649014 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:07.648985 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15c278e5-f3a7-438a-8f98-d0acbadf2484","Type":"ContainerStarted","Data":"a5e325338aca107a64c466307f3288236572dc9915fe7cd1438d845b263c073a"} Jul 10 19:02:07.649454 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:07.649070 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15c278e5-f3a7-438a-8f98-d0acbadf2484","Type":"ContainerStarted","Data":"590ebaeb087ffd9986d021414fe3237d67beedcfca440a750a83fe86adc9520b"} Jul 10 19:02:07.649454 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:07.649130 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15c278e5-f3a7-438a-8f98-d0acbadf2484","Type":"ContainerStarted","Data":"f9b843a80e524772a5b443bfef89b1f0864ffb123155a3f0442d26daff68c04e"} Jul 10 19:02:08.656545 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:08.656509 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15c278e5-f3a7-438a-8f98-d0acbadf2484","Type":"ContainerStarted","Data":"3795300ec5cc9539dfe7443d97b1653925dfb2b297afb3870066ffdfaba3dcdb"} Jul 10 19:02:08.691205 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:08.691158 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=9.220912253 podStartE2EDuration="17.691139673s" totalImagesPullingTime="8.437074242s" totalInitContainerRuntime="33.153178ms" isStatefulPod=false podCreationTimestamp="2026-07-10 19:01:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:02:08.688086592 +0000 UTC m=+110.224429024" watchObservedRunningTime="2026-07-10 19:02:08.691139673 +0000 UTC m=+110.227482104" Jul 10 19:02:09.938634 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:09.938595 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:02:09.938994 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:09.938661 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:02:11.819540 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:11.819508 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:02:13.702267 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:13.702232 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:13.702737 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:13.702290 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:13.707117 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:13.707079 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:14.684828 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:14.684798 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:02:15.779378 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:15.779319 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b78bb8dd5-xjcjp" podUID="e49ee3e6-1da9-49bc-8fc4-5dfcd9250982" containerName="console" containerID="cri-o://58b2c3538dc56b7c98c54c47319293c7e91a923a7a45abddae27d5ea421ee347" gracePeriod=15 Jul 10 19:02:16.061618 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.061566 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:02:16.147073 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.147046 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-oauth-serving-cert\") pod \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " Jul 10 19:02:16.147199 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.147081 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-trusted-ca-bundle\") pod \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " Jul 10 19:02:16.147243 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.147220 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwktj\" (UniqueName: \"kubernetes.io/projected/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-kube-api-access-vwktj\") pod \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " Jul 10 19:02:16.147278 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.147264 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-console-oauth-config\") pod \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " Jul 10 19:02:16.147535 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.147299 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-service-ca\") pod \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " Jul 10 19:02:16.147535 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.147316 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-console-config\") pod \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " Jul 10 19:02:16.147535 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.147362 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-console-serving-cert\") pod \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\" (UID: \"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982\") " Jul 10 19:02:16.147535 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.147426 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e49ee3e6-1da9-49bc-8fc4-5dfcd9250982" (UID: "e49ee3e6-1da9-49bc-8fc4-5dfcd9250982"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:02:16.147535 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.147442 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e49ee3e6-1da9-49bc-8fc4-5dfcd9250982" (UID: "e49ee3e6-1da9-49bc-8fc4-5dfcd9250982"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:02:16.147809 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.147687 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-oauth-serving-cert\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:16.147809 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.147706 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-trusted-ca-bundle\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:16.147809 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.147733 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-console-config" (OuterVolumeSpecName: "console-config") pod "e49ee3e6-1da9-49bc-8fc4-5dfcd9250982" (UID: "e49ee3e6-1da9-49bc-8fc4-5dfcd9250982"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:02:16.147809 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.147739 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-service-ca" (OuterVolumeSpecName: "service-ca") pod "e49ee3e6-1da9-49bc-8fc4-5dfcd9250982" (UID: "e49ee3e6-1da9-49bc-8fc4-5dfcd9250982"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:02:16.149432 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.149413 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e49ee3e6-1da9-49bc-8fc4-5dfcd9250982" (UID: "e49ee3e6-1da9-49bc-8fc4-5dfcd9250982"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:02:16.149518 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.149432 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e49ee3e6-1da9-49bc-8fc4-5dfcd9250982" (UID: "e49ee3e6-1da9-49bc-8fc4-5dfcd9250982"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:02:16.149518 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.149436 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-kube-api-access-vwktj" (OuterVolumeSpecName: "kube-api-access-vwktj") pod "e49ee3e6-1da9-49bc-8fc4-5dfcd9250982" (UID: "e49ee3e6-1da9-49bc-8fc4-5dfcd9250982"). InnerVolumeSpecName "kube-api-access-vwktj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:02:16.248544 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.248523 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwktj\" (UniqueName: \"kubernetes.io/projected/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-kube-api-access-vwktj\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:16.248544 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.248544 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-console-oauth-config\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:16.248675 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.248554 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:16.248675 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.248563 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-console-config\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:16.248675 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.248571 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982-console-serving-cert\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:16.695320 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.695288 2571 generic.go:356] "Generic (PLEG): container finished" podID="e49ee3e6-1da9-49bc-8fc4-5dfcd9250982" containerID="58b2c3538dc56b7c98c54c47319293c7e91a923a7a45abddae27d5ea421ee347" exitCode=0 Jul 10 19:02:16.695594 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.695343 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b78bb8dd5-xjcjp" Jul 10 19:02:16.695594 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.695358 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b78bb8dd5-xjcjp" event={"ID":"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982","Type":"ContainerDied","Data":"58b2c3538dc56b7c98c54c47319293c7e91a923a7a45abddae27d5ea421ee347"} Jul 10 19:02:16.695594 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.695440 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b78bb8dd5-xjcjp" event={"ID":"e49ee3e6-1da9-49bc-8fc4-5dfcd9250982","Type":"ContainerDied","Data":"467c94a865b2d98beab8ee2289acfcfca0518dbb6e4610e080352fb3f42b7543"} Jul 10 19:02:16.695594 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.695467 2571 scope.go:117] "RemoveContainer" containerID="58b2c3538dc56b7c98c54c47319293c7e91a923a7a45abddae27d5ea421ee347" Jul 10 19:02:16.697155 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.697129 2571 generic.go:356] "Generic (PLEG): container finished" podID="6ed7ba8a-9eab-45ed-944c-fd0c5d905551" containerID="45b1290e8b898f545732e82ebc71119ff46baa87159ec05afae46077847f0e23" exitCode=0 Jul 10 19:02:16.697283 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.697196 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" event={"ID":"6ed7ba8a-9eab-45ed-944c-fd0c5d905551","Type":"ContainerDied","Data":"45b1290e8b898f545732e82ebc71119ff46baa87159ec05afae46077847f0e23"} Jul 10 19:02:16.697535 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.697521 2571 scope.go:117] "RemoveContainer" containerID="45b1290e8b898f545732e82ebc71119ff46baa87159ec05afae46077847f0e23" Jul 10 19:02:16.706360 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.706346 2571 scope.go:117] "RemoveContainer" containerID="58b2c3538dc56b7c98c54c47319293c7e91a923a7a45abddae27d5ea421ee347" Jul 10 19:02:16.706756 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:02:16.706737 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58b2c3538dc56b7c98c54c47319293c7e91a923a7a45abddae27d5ea421ee347\": container with ID starting with 58b2c3538dc56b7c98c54c47319293c7e91a923a7a45abddae27d5ea421ee347 not found: ID does not exist" containerID="58b2c3538dc56b7c98c54c47319293c7e91a923a7a45abddae27d5ea421ee347" Jul 10 19:02:16.706821 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.706763 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58b2c3538dc56b7c98c54c47319293c7e91a923a7a45abddae27d5ea421ee347"} err="failed to get container status \"58b2c3538dc56b7c98c54c47319293c7e91a923a7a45abddae27d5ea421ee347\": rpc error: code = NotFound desc = could not find container \"58b2c3538dc56b7c98c54c47319293c7e91a923a7a45abddae27d5ea421ee347\": container with ID starting with 58b2c3538dc56b7c98c54c47319293c7e91a923a7a45abddae27d5ea421ee347 not found: ID does not exist" Jul 10 19:02:16.742421 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.742391 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b78bb8dd5-xjcjp"] Jul 10 19:02:16.744151 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:16.744127 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b78bb8dd5-xjcjp"] Jul 10 19:02:17.017152 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:17.017052 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e49ee3e6-1da9-49bc-8fc4-5dfcd9250982" path="/var/lib/kubelet/pods/e49ee3e6-1da9-49bc-8fc4-5dfcd9250982/volumes" Jul 10 19:02:17.705819 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:17.705773 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7h2bn" event={"ID":"6ed7ba8a-9eab-45ed-944c-fd0c5d905551","Type":"ContainerStarted","Data":"1ae1ce41ac3de8e56378e1722877ed539f75de157adb535c8c19f3377723d671"} Jul 10 19:02:22.361301 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.361264 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" podUID="5501e15d-59e4-4a41-8cb0-77ea136593ca" containerName="registry" containerID="cri-o://5f3b07163d968face513f365c34a3200e6c9a3a0045a20ce67ed6f937a478908" gracePeriod=30 Jul 10 19:02:22.641129 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.641091 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:02:22.704499 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.704476 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5501e15d-59e4-4a41-8cb0-77ea136593ca-installation-pull-secrets\") pod \"5501e15d-59e4-4a41-8cb0-77ea136593ca\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " Jul 10 19:02:22.704619 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.704525 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5501e15d-59e4-4a41-8cb0-77ea136593ca-image-registry-private-configuration\") pod \"5501e15d-59e4-4a41-8cb0-77ea136593ca\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " Jul 10 19:02:22.704619 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.704557 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls\") pod \"5501e15d-59e4-4a41-8cb0-77ea136593ca\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " Jul 10 19:02:22.704619 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.704596 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-bound-sa-token\") pod \"5501e15d-59e4-4a41-8cb0-77ea136593ca\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " Jul 10 19:02:22.704756 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.704643 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5501e15d-59e4-4a41-8cb0-77ea136593ca-trusted-ca\") pod \"5501e15d-59e4-4a41-8cb0-77ea136593ca\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " Jul 10 19:02:22.704809 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.704748 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-certificates\") pod \"5501e15d-59e4-4a41-8cb0-77ea136593ca\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " Jul 10 19:02:22.704906 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.704892 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5501e15d-59e4-4a41-8cb0-77ea136593ca-ca-trust-extracted\") pod \"5501e15d-59e4-4a41-8cb0-77ea136593ca\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " Jul 10 19:02:22.704966 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.704927 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5d26\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-kube-api-access-b5d26\") pod \"5501e15d-59e4-4a41-8cb0-77ea136593ca\" (UID: \"5501e15d-59e4-4a41-8cb0-77ea136593ca\") " Jul 10 19:02:22.705200 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.705168 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5501e15d-59e4-4a41-8cb0-77ea136593ca-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5501e15d-59e4-4a41-8cb0-77ea136593ca" (UID: "5501e15d-59e4-4a41-8cb0-77ea136593ca"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:02:22.705312 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.705224 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5501e15d-59e4-4a41-8cb0-77ea136593ca" (UID: "5501e15d-59e4-4a41-8cb0-77ea136593ca"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:02:22.705377 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.705336 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5501e15d-59e4-4a41-8cb0-77ea136593ca-trusted-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:22.707503 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.707468 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-kube-api-access-b5d26" (OuterVolumeSpecName: "kube-api-access-b5d26") pod "5501e15d-59e4-4a41-8cb0-77ea136593ca" (UID: "5501e15d-59e4-4a41-8cb0-77ea136593ca"). InnerVolumeSpecName "kube-api-access-b5d26". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:02:22.707609 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.707533 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5501e15d-59e4-4a41-8cb0-77ea136593ca-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5501e15d-59e4-4a41-8cb0-77ea136593ca" (UID: "5501e15d-59e4-4a41-8cb0-77ea136593ca"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:02:22.707609 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.707558 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5501e15d-59e4-4a41-8cb0-77ea136593ca" (UID: "5501e15d-59e4-4a41-8cb0-77ea136593ca"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:02:22.707725 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.707684 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5501e15d-59e4-4a41-8cb0-77ea136593ca" (UID: "5501e15d-59e4-4a41-8cb0-77ea136593ca"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:02:22.707725 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.707716 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5501e15d-59e4-4a41-8cb0-77ea136593ca-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5501e15d-59e4-4a41-8cb0-77ea136593ca" (UID: "5501e15d-59e4-4a41-8cb0-77ea136593ca"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:02:22.713784 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.713759 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5501e15d-59e4-4a41-8cb0-77ea136593ca-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5501e15d-59e4-4a41-8cb0-77ea136593ca" (UID: "5501e15d-59e4-4a41-8cb0-77ea136593ca"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:02:22.727684 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.727659 2571 generic.go:356] "Generic (PLEG): container finished" podID="5501e15d-59e4-4a41-8cb0-77ea136593ca" containerID="5f3b07163d968face513f365c34a3200e6c9a3a0045a20ce67ed6f937a478908" exitCode=0 Jul 10 19:02:22.727782 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.727717 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" Jul 10 19:02:22.727782 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.727744 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" event={"ID":"5501e15d-59e4-4a41-8cb0-77ea136593ca","Type":"ContainerDied","Data":"5f3b07163d968face513f365c34a3200e6c9a3a0045a20ce67ed6f937a478908"} Jul 10 19:02:22.727886 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.727795 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644b746fdc-rjsj4" event={"ID":"5501e15d-59e4-4a41-8cb0-77ea136593ca","Type":"ContainerDied","Data":"e6c5f2ded7a0c3c09da9f4ea910d0c0a2d127ce051dbfc2deb934aed11106447"} Jul 10 19:02:22.727886 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.727815 2571 scope.go:117] "RemoveContainer" containerID="5f3b07163d968face513f365c34a3200e6c9a3a0045a20ce67ed6f937a478908" Jul 10 19:02:22.736053 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.736036 2571 scope.go:117] "RemoveContainer" containerID="5f3b07163d968face513f365c34a3200e6c9a3a0045a20ce67ed6f937a478908" Jul 10 19:02:22.736327 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:02:22.736310 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f3b07163d968face513f365c34a3200e6c9a3a0045a20ce67ed6f937a478908\": container with ID starting with 5f3b07163d968face513f365c34a3200e6c9a3a0045a20ce67ed6f937a478908 not found: ID does not exist" containerID="5f3b07163d968face513f365c34a3200e6c9a3a0045a20ce67ed6f937a478908" Jul 10 19:02:22.736382 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.736335 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f3b07163d968face513f365c34a3200e6c9a3a0045a20ce67ed6f937a478908"} err="failed to get container status \"5f3b07163d968face513f365c34a3200e6c9a3a0045a20ce67ed6f937a478908\": rpc error: code = NotFound desc = could not find container \"5f3b07163d968face513f365c34a3200e6c9a3a0045a20ce67ed6f937a478908\": container with ID starting with 5f3b07163d968face513f365c34a3200e6c9a3a0045a20ce67ed6f937a478908 not found: ID does not exist" Jul 10 19:02:22.753014 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.752991 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-644b746fdc-rjsj4"] Jul 10 19:02:22.760713 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.760693 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-644b746fdc-rjsj4"] Jul 10 19:02:22.806614 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.806592 2571 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-certificates\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:22.806614 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.806612 2571 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5501e15d-59e4-4a41-8cb0-77ea136593ca-ca-trust-extracted\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:22.806716 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.806623 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5d26\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-kube-api-access-b5d26\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:22.806716 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.806632 2571 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5501e15d-59e4-4a41-8cb0-77ea136593ca-installation-pull-secrets\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:22.806716 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.806640 2571 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5501e15d-59e4-4a41-8cb0-77ea136593ca-image-registry-private-configuration\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:22.806716 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.806650 2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-registry-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:22.806716 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:22.806658 2571 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5501e15d-59e4-4a41-8cb0-77ea136593ca-bound-sa-token\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:23.016942 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:23.016860 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5501e15d-59e4-4a41-8cb0-77ea136593ca" path="/var/lib/kubelet/pods/5501e15d-59e4-4a41-8cb0-77ea136593ca/volumes" Jul 10 19:02:26.514568 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.514510 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c57df6cf8-jpsmh" podUID="bd20bbb3-4733-4ea4-b5e8-d7fce01202b7" containerName="console" containerID="cri-o://53e719f201e7e6427bf4e21eca920ab99aad21571d49ef1fcf1bc5ad442e6e1a" gracePeriod=15 Jul 10 19:02:26.752531 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.752502 2571 generic.go:356] "Generic (PLEG): container finished" podID="bd20bbb3-4733-4ea4-b5e8-d7fce01202b7" containerID="53e719f201e7e6427bf4e21eca920ab99aad21571d49ef1fcf1bc5ad442e6e1a" exitCode=0 Jul 10 19:02:26.752695 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.752578 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c57df6cf8-jpsmh" event={"ID":"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7","Type":"ContainerDied","Data":"53e719f201e7e6427bf4e21eca920ab99aad21571d49ef1fcf1bc5ad442e6e1a"} Jul 10 19:02:26.802026 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.802001 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:02:26.841171 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.841148 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-service-ca\") pod \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " Jul 10 19:02:26.841327 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.841183 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-oauth-serving-cert\") pod \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " Jul 10 19:02:26.841327 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.841214 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsxbc\" (UniqueName: \"kubernetes.io/projected/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-kube-api-access-zsxbc\") pod \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " Jul 10 19:02:26.841327 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.841273 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-console-oauth-config\") pod \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " Jul 10 19:02:26.841327 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.841296 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-console-config\") pod \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " Jul 10 19:02:26.841327 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.841315 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-console-serving-cert\") pod \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\" (UID: \"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7\") " Jul 10 19:02:26.841738 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.841614 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bd20bbb3-4733-4ea4-b5e8-d7fce01202b7" (UID: "bd20bbb3-4733-4ea4-b5e8-d7fce01202b7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:02:26.841850 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.841742 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-service-ca" (OuterVolumeSpecName: "service-ca") pod "bd20bbb3-4733-4ea4-b5e8-d7fce01202b7" (UID: "bd20bbb3-4733-4ea4-b5e8-d7fce01202b7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:02:26.841850 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.841759 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-console-config" (OuterVolumeSpecName: "console-config") pod "bd20bbb3-4733-4ea4-b5e8-d7fce01202b7" (UID: "bd20bbb3-4733-4ea4-b5e8-d7fce01202b7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:02:26.843398 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.843373 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-kube-api-access-zsxbc" (OuterVolumeSpecName: "kube-api-access-zsxbc") pod "bd20bbb3-4733-4ea4-b5e8-d7fce01202b7" (UID: "bd20bbb3-4733-4ea4-b5e8-d7fce01202b7"). InnerVolumeSpecName "kube-api-access-zsxbc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:02:26.843486 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.843449 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bd20bbb3-4733-4ea4-b5e8-d7fce01202b7" (UID: "bd20bbb3-4733-4ea4-b5e8-d7fce01202b7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:02:26.843762 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.843744 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bd20bbb3-4733-4ea4-b5e8-d7fce01202b7" (UID: "bd20bbb3-4733-4ea4-b5e8-d7fce01202b7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:02:26.942215 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.942185 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsxbc\" (UniqueName: \"kubernetes.io/projected/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-kube-api-access-zsxbc\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:26.942215 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.942209 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-console-oauth-config\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:26.942215 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.942219 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-console-config\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:26.942409 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.942230 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-console-serving-cert\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:26.942409 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.942239 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:26.942409 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:26.942247 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7-oauth-serving-cert\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:02:27.758303 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:27.758273 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c57df6cf8-jpsmh" Jul 10 19:02:27.758303 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:27.758288 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c57df6cf8-jpsmh" event={"ID":"bd20bbb3-4733-4ea4-b5e8-d7fce01202b7","Type":"ContainerDied","Data":"e0036b7b1eaa8c29d9cfe96e11e297c2a5d2fed381ff42a47452ae1bbbb1b278"} Jul 10 19:02:27.758767 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:27.758342 2571 scope.go:117] "RemoveContainer" containerID="53e719f201e7e6427bf4e21eca920ab99aad21571d49ef1fcf1bc5ad442e6e1a" Jul 10 19:02:27.777338 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:27.777309 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c57df6cf8-jpsmh"] Jul 10 19:02:27.782984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:27.782961 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c57df6cf8-jpsmh"] Jul 10 19:02:28.861056 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:28.861025 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs\") pod \"network-metrics-daemon-9p2rq\" (UID: \"f911a124-c338-40f4-983f-c61a0d899d93\") " pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:02:28.863324 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:28.863300 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911a124-c338-40f4-983f-c61a0d899d93-metrics-certs\") pod \"network-metrics-daemon-9p2rq\" (UID: \"f911a124-c338-40f4-983f-c61a0d899d93\") " pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:02:29.019434 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:29.019403 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd20bbb3-4733-4ea4-b5e8-d7fce01202b7" path="/var/lib/kubelet/pods/bd20bbb3-4733-4ea4-b5e8-d7fce01202b7/volumes" Jul 10 19:02:29.133334 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:29.133260 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zb6nn\"" Jul 10 19:02:29.140860 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:29.140837 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9p2rq" Jul 10 19:02:29.343617 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:29.333208 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9p2rq"] Jul 10 19:02:29.345466 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:02:29.345439 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf911a124_c338_40f4_983f_c61a0d899d93.slice/crio-f8aff385a991e6da661fac9e1ed35b1a12ed28f77681b73827b3ce4e65e07dfb WatchSource:0}: Error finding container f8aff385a991e6da661fac9e1ed35b1a12ed28f77681b73827b3ce4e65e07dfb: Status 404 returned error can't find the container with id f8aff385a991e6da661fac9e1ed35b1a12ed28f77681b73827b3ce4e65e07dfb Jul 10 19:02:29.772128 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:29.772074 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9p2rq" event={"ID":"f911a124-c338-40f4-983f-c61a0d899d93","Type":"ContainerStarted","Data":"f8aff385a991e6da661fac9e1ed35b1a12ed28f77681b73827b3ce4e65e07dfb"} Jul 10 19:02:29.944943 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:29.944911 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:02:29.950476 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:29.950443 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-78b7bbd787-cstjx" Jul 10 19:02:31.783399 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:31.783369 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9p2rq" event={"ID":"f911a124-c338-40f4-983f-c61a0d899d93","Type":"ContainerStarted","Data":"50a4a648aa0afbba2231bf330e92c7ca24eba53dbbda12af09f1e76f02f0b18e"} Jul 10 19:02:31.783784 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:31.783427 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9p2rq" event={"ID":"f911a124-c338-40f4-983f-c61a0d899d93","Type":"ContainerStarted","Data":"3af5d3824e5b02cab0086932a3890fb1c83320e24e40483dad99456a128332fd"} Jul 10 19:02:31.805067 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:31.802627 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9p2rq" podStartSLOduration=131.260594422 podStartE2EDuration="2m12.802605555s" totalImagesPullingTime="1.542011133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:02:31.801542842 +0000 UTC m=+133.337885278" watchObservedRunningTime="2026-07-10 19:02:31.802605555 +0000 UTC m=+133.338947989" Jul 10 19:02:51.819743 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:51.819695 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:02:51.835558 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:51.835534 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:02:51.884263 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:02:51.884235 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:05.578733 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:05.578694 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 19:03:05.579160 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:05.579082 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="alertmanager" containerID="cri-o://941dc1333d8adb2ec9fe3696ded34f0d7e12879673aab5349d6a810b4690a655" gracePeriod=120 Jul 10 19:03:05.579160 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:05.579111 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="kube-rbac-proxy-metric" containerID="cri-o://1b7fa350416c7620b651f2c524385d85c2d41a22b38fad750bf9ae979cef5b0f" gracePeriod=120 Jul 10 19:03:05.579301 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:05.579153 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="kube-rbac-proxy-web" containerID="cri-o://7b1f88ab126f1d8f3d2439937d0378e303ec11cbbf367ceb5b15818a875232aa" gracePeriod=120 Jul 10 19:03:05.579301 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:05.579176 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="prom-label-proxy" containerID="cri-o://f993869d344244ed8ffe53536c3b28702130b0360c316fee9443277677f5493a" gracePeriod=120 Jul 10 19:03:05.579301 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:05.579225 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="kube-rbac-proxy" containerID="cri-o://dd8c352cd70202f955612aa8059fd5b2d8cdd449d5ed41d08f05e2b7bc020176" gracePeriod=120 Jul 10 19:03:05.579301 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:05.579204 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="config-reloader" containerID="cri-o://1c4e9e43a1b476d5b55f96997a042cf63e2023eb90694d2651f319da7ce291e9" gracePeriod=120 Jul 10 19:03:05.924456 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:05.924377 2571 generic.go:356] "Generic (PLEG): container finished" podID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerID="f993869d344244ed8ffe53536c3b28702130b0360c316fee9443277677f5493a" exitCode=0 Jul 10 19:03:05.924456 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:05.924405 2571 generic.go:356] "Generic (PLEG): container finished" podID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerID="1b7fa350416c7620b651f2c524385d85c2d41a22b38fad750bf9ae979cef5b0f" exitCode=0 Jul 10 19:03:05.924456 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:05.924415 2571 generic.go:356] "Generic (PLEG): container finished" podID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerID="dd8c352cd70202f955612aa8059fd5b2d8cdd449d5ed41d08f05e2b7bc020176" exitCode=0 Jul 10 19:03:05.924456 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:05.924423 2571 generic.go:356] "Generic (PLEG): container finished" podID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerID="1c4e9e43a1b476d5b55f96997a042cf63e2023eb90694d2651f319da7ce291e9" exitCode=0 Jul 10 19:03:05.924456 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:05.924430 2571 generic.go:356] "Generic (PLEG): container finished" podID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerID="941dc1333d8adb2ec9fe3696ded34f0d7e12879673aab5349d6a810b4690a655" exitCode=0 Jul 10 19:03:05.924456 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:05.924449 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"627e86ba-0df2-4638-a56c-d9f990f4ff36","Type":"ContainerDied","Data":"f993869d344244ed8ffe53536c3b28702130b0360c316fee9443277677f5493a"} Jul 10 19:03:05.924721 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:05.924510 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"627e86ba-0df2-4638-a56c-d9f990f4ff36","Type":"ContainerDied","Data":"1b7fa350416c7620b651f2c524385d85c2d41a22b38fad750bf9ae979cef5b0f"} Jul 10 19:03:05.924721 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:05.924539 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"627e86ba-0df2-4638-a56c-d9f990f4ff36","Type":"ContainerDied","Data":"dd8c352cd70202f955612aa8059fd5b2d8cdd449d5ed41d08f05e2b7bc020176"} Jul 10 19:03:05.924721 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:05.924564 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"627e86ba-0df2-4638-a56c-d9f990f4ff36","Type":"ContainerDied","Data":"1c4e9e43a1b476d5b55f96997a042cf63e2023eb90694d2651f319da7ce291e9"} Jul 10 19:03:05.924721 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:05.924587 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"627e86ba-0df2-4638-a56c-d9f990f4ff36","Type":"ContainerDied","Data":"941dc1333d8adb2ec9fe3696ded34f0d7e12879673aab5349d6a810b4690a655"} Jul 10 19:03:06.817886 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.817864 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:06.889922 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.889893 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-config-volume\") pod \"627e86ba-0df2-4638-a56c-d9f990f4ff36\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " Jul 10 19:03:06.890090 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.889934 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/627e86ba-0df2-4638-a56c-d9f990f4ff36-metrics-client-ca\") pod \"627e86ba-0df2-4638-a56c-d9f990f4ff36\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " Jul 10 19:03:06.890090 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.889958 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-kube-rbac-proxy-web\") pod \"627e86ba-0df2-4638-a56c-d9f990f4ff36\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " Jul 10 19:03:06.890090 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.889976 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/627e86ba-0df2-4638-a56c-d9f990f4ff36-config-out\") pod \"627e86ba-0df2-4638-a56c-d9f990f4ff36\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " Jul 10 19:03:06.890090 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.890002 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhp94\" (UniqueName: \"kubernetes.io/projected/627e86ba-0df2-4638-a56c-d9f990f4ff36-kube-api-access-fhp94\") pod \"627e86ba-0df2-4638-a56c-d9f990f4ff36\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " Jul 10 19:03:06.890090 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.890025 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-cluster-tls-config\") pod \"627e86ba-0df2-4638-a56c-d9f990f4ff36\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " Jul 10 19:03:06.890090 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.890051 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-kube-rbac-proxy\") pod \"627e86ba-0df2-4638-a56c-d9f990f4ff36\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " Jul 10 19:03:06.890446 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.890090 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-main-tls\") pod \"627e86ba-0df2-4638-a56c-d9f990f4ff36\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " Jul 10 19:03:06.890446 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.890172 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/627e86ba-0df2-4638-a56c-d9f990f4ff36-alertmanager-main-db\") pod \"627e86ba-0df2-4638-a56c-d9f990f4ff36\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " Jul 10 19:03:06.890446 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.890214 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-web-config\") pod \"627e86ba-0df2-4638-a56c-d9f990f4ff36\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " Jul 10 19:03:06.890446 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.890247 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/627e86ba-0df2-4638-a56c-d9f990f4ff36-alertmanager-trusted-ca-bundle\") pod \"627e86ba-0df2-4638-a56c-d9f990f4ff36\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " Jul 10 19:03:06.890446 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.890281 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/627e86ba-0df2-4638-a56c-d9f990f4ff36-tls-assets\") pod \"627e86ba-0df2-4638-a56c-d9f990f4ff36\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " Jul 10 19:03:06.890446 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.890325 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-kube-rbac-proxy-metric\") pod \"627e86ba-0df2-4638-a56c-d9f990f4ff36\" (UID: \"627e86ba-0df2-4638-a56c-d9f990f4ff36\") " Jul 10 19:03:06.891035 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.890882 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/627e86ba-0df2-4638-a56c-d9f990f4ff36-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "627e86ba-0df2-4638-a56c-d9f990f4ff36" (UID: "627e86ba-0df2-4638-a56c-d9f990f4ff36"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:03:06.891156 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.891059 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/627e86ba-0df2-4638-a56c-d9f990f4ff36-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "627e86ba-0df2-4638-a56c-d9f990f4ff36" (UID: "627e86ba-0df2-4638-a56c-d9f990f4ff36"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:03:06.891479 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.891446 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/627e86ba-0df2-4638-a56c-d9f990f4ff36-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "627e86ba-0df2-4638-a56c-d9f990f4ff36" (UID: "627e86ba-0df2-4638-a56c-d9f990f4ff36"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:03:06.893443 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.893304 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "627e86ba-0df2-4638-a56c-d9f990f4ff36" (UID: "627e86ba-0df2-4638-a56c-d9f990f4ff36"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:03:06.893443 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.893416 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/627e86ba-0df2-4638-a56c-d9f990f4ff36-kube-api-access-fhp94" (OuterVolumeSpecName: "kube-api-access-fhp94") pod "627e86ba-0df2-4638-a56c-d9f990f4ff36" (UID: "627e86ba-0df2-4638-a56c-d9f990f4ff36"). InnerVolumeSpecName "kube-api-access-fhp94". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:03:06.893874 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.893852 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/627e86ba-0df2-4638-a56c-d9f990f4ff36-config-out" (OuterVolumeSpecName: "config-out") pod "627e86ba-0df2-4638-a56c-d9f990f4ff36" (UID: "627e86ba-0df2-4638-a56c-d9f990f4ff36"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:03:06.894010 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.893988 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/627e86ba-0df2-4638-a56c-d9f990f4ff36-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "627e86ba-0df2-4638-a56c-d9f990f4ff36" (UID: "627e86ba-0df2-4638-a56c-d9f990f4ff36"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:03:06.894137 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.894051 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-config-volume" (OuterVolumeSpecName: "config-volume") pod "627e86ba-0df2-4638-a56c-d9f990f4ff36" (UID: "627e86ba-0df2-4638-a56c-d9f990f4ff36"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:03:06.894828 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.894796 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "627e86ba-0df2-4638-a56c-d9f990f4ff36" (UID: "627e86ba-0df2-4638-a56c-d9f990f4ff36"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:03:06.895323 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.895290 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "627e86ba-0df2-4638-a56c-d9f990f4ff36" (UID: "627e86ba-0df2-4638-a56c-d9f990f4ff36"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:03:06.895976 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.895946 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "627e86ba-0df2-4638-a56c-d9f990f4ff36" (UID: "627e86ba-0df2-4638-a56c-d9f990f4ff36"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:03:06.898060 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.898008 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "627e86ba-0df2-4638-a56c-d9f990f4ff36" (UID: "627e86ba-0df2-4638-a56c-d9f990f4ff36"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:03:06.905708 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.905686 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-web-config" (OuterVolumeSpecName: "web-config") pod "627e86ba-0df2-4638-a56c-d9f990f4ff36" (UID: "627e86ba-0df2-4638-a56c-d9f990f4ff36"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:03:06.931194 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.931169 2571 generic.go:356] "Generic (PLEG): container finished" podID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerID="7b1f88ab126f1d8f3d2439937d0378e303ec11cbbf367ceb5b15818a875232aa" exitCode=0 Jul 10 19:03:06.931282 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.931259 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:06.931282 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.931257 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"627e86ba-0df2-4638-a56c-d9f990f4ff36","Type":"ContainerDied","Data":"7b1f88ab126f1d8f3d2439937d0378e303ec11cbbf367ceb5b15818a875232aa"} Jul 10 19:03:06.931347 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.931329 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"627e86ba-0df2-4638-a56c-d9f990f4ff36","Type":"ContainerDied","Data":"7ba4a0f6e10d46e421058f102e21d86b30373d117e5507913734375677afcfc2"} Jul 10 19:03:06.931379 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.931363 2571 scope.go:117] "RemoveContainer" containerID="f993869d344244ed8ffe53536c3b28702130b0360c316fee9443277677f5493a" Jul 10 19:03:06.939646 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.939565 2571 scope.go:117] "RemoveContainer" containerID="1b7fa350416c7620b651f2c524385d85c2d41a22b38fad750bf9ae979cef5b0f" Jul 10 19:03:06.946270 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.946191 2571 scope.go:117] "RemoveContainer" containerID="dd8c352cd70202f955612aa8059fd5b2d8cdd449d5ed41d08f05e2b7bc020176" Jul 10 19:03:06.954158 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.954141 2571 scope.go:117] "RemoveContainer" containerID="7b1f88ab126f1d8f3d2439937d0378e303ec11cbbf367ceb5b15818a875232aa" Jul 10 19:03:06.960092 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.960069 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 19:03:06.962022 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.962005 2571 scope.go:117] "RemoveContainer" containerID="1c4e9e43a1b476d5b55f96997a042cf63e2023eb90694d2651f319da7ce291e9" Jul 10 19:03:06.966430 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.966412 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 19:03:06.969719 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.969703 2571 scope.go:117] "RemoveContainer" containerID="941dc1333d8adb2ec9fe3696ded34f0d7e12879673aab5349d6a810b4690a655" Jul 10 19:03:06.976359 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.976344 2571 scope.go:117] "RemoveContainer" containerID="44a17d61cad01ac43dd6752194bc8e0e258eea55292c5988c18800c9f5977847" Jul 10 19:03:06.982874 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.982853 2571 scope.go:117] "RemoveContainer" containerID="f993869d344244ed8ffe53536c3b28702130b0360c316fee9443277677f5493a" Jul 10 19:03:06.983161 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:06.983139 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f993869d344244ed8ffe53536c3b28702130b0360c316fee9443277677f5493a\": container with ID starting with f993869d344244ed8ffe53536c3b28702130b0360c316fee9443277677f5493a not found: ID does not exist" containerID="f993869d344244ed8ffe53536c3b28702130b0360c316fee9443277677f5493a" Jul 10 19:03:06.983232 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.983171 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f993869d344244ed8ffe53536c3b28702130b0360c316fee9443277677f5493a"} err="failed to get container status \"f993869d344244ed8ffe53536c3b28702130b0360c316fee9443277677f5493a\": rpc error: code = NotFound desc = could not find container \"f993869d344244ed8ffe53536c3b28702130b0360c316fee9443277677f5493a\": container with ID starting with f993869d344244ed8ffe53536c3b28702130b0360c316fee9443277677f5493a not found: ID does not exist" Jul 10 19:03:06.983232 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.983189 2571 scope.go:117] "RemoveContainer" containerID="1b7fa350416c7620b651f2c524385d85c2d41a22b38fad750bf9ae979cef5b0f" Jul 10 19:03:06.983426 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:06.983410 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b7fa350416c7620b651f2c524385d85c2d41a22b38fad750bf9ae979cef5b0f\": container with ID starting with 1b7fa350416c7620b651f2c524385d85c2d41a22b38fad750bf9ae979cef5b0f not found: ID does not exist" containerID="1b7fa350416c7620b651f2c524385d85c2d41a22b38fad750bf9ae979cef5b0f" Jul 10 19:03:06.983469 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.983434 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b7fa350416c7620b651f2c524385d85c2d41a22b38fad750bf9ae979cef5b0f"} err="failed to get container status \"1b7fa350416c7620b651f2c524385d85c2d41a22b38fad750bf9ae979cef5b0f\": rpc error: code = NotFound desc = could not find container \"1b7fa350416c7620b651f2c524385d85c2d41a22b38fad750bf9ae979cef5b0f\": container with ID starting with 1b7fa350416c7620b651f2c524385d85c2d41a22b38fad750bf9ae979cef5b0f not found: ID does not exist" Jul 10 19:03:06.983469 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.983449 2571 scope.go:117] "RemoveContainer" containerID="dd8c352cd70202f955612aa8059fd5b2d8cdd449d5ed41d08f05e2b7bc020176" Jul 10 19:03:06.983693 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:06.983673 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd8c352cd70202f955612aa8059fd5b2d8cdd449d5ed41d08f05e2b7bc020176\": container with ID starting with dd8c352cd70202f955612aa8059fd5b2d8cdd449d5ed41d08f05e2b7bc020176 not found: ID does not exist" containerID="dd8c352cd70202f955612aa8059fd5b2d8cdd449d5ed41d08f05e2b7bc020176" Jul 10 19:03:06.983738 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.983704 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd8c352cd70202f955612aa8059fd5b2d8cdd449d5ed41d08f05e2b7bc020176"} err="failed to get container status \"dd8c352cd70202f955612aa8059fd5b2d8cdd449d5ed41d08f05e2b7bc020176\": rpc error: code = NotFound desc = could not find container \"dd8c352cd70202f955612aa8059fd5b2d8cdd449d5ed41d08f05e2b7bc020176\": container with ID starting with dd8c352cd70202f955612aa8059fd5b2d8cdd449d5ed41d08f05e2b7bc020176 not found: ID does not exist" Jul 10 19:03:06.983738 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.983720 2571 scope.go:117] "RemoveContainer" containerID="7b1f88ab126f1d8f3d2439937d0378e303ec11cbbf367ceb5b15818a875232aa" Jul 10 19:03:06.983929 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:06.983912 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b1f88ab126f1d8f3d2439937d0378e303ec11cbbf367ceb5b15818a875232aa\": container with ID starting with 7b1f88ab126f1d8f3d2439937d0378e303ec11cbbf367ceb5b15818a875232aa not found: ID does not exist" containerID="7b1f88ab126f1d8f3d2439937d0378e303ec11cbbf367ceb5b15818a875232aa" Jul 10 19:03:06.983968 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.983934 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b1f88ab126f1d8f3d2439937d0378e303ec11cbbf367ceb5b15818a875232aa"} err="failed to get container status \"7b1f88ab126f1d8f3d2439937d0378e303ec11cbbf367ceb5b15818a875232aa\": rpc error: code = NotFound desc = could not find container \"7b1f88ab126f1d8f3d2439937d0378e303ec11cbbf367ceb5b15818a875232aa\": container with ID starting with 7b1f88ab126f1d8f3d2439937d0378e303ec11cbbf367ceb5b15818a875232aa not found: ID does not exist" Jul 10 19:03:06.983968 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.983948 2571 scope.go:117] "RemoveContainer" containerID="1c4e9e43a1b476d5b55f96997a042cf63e2023eb90694d2651f319da7ce291e9" Jul 10 19:03:06.984197 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:06.984177 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c4e9e43a1b476d5b55f96997a042cf63e2023eb90694d2651f319da7ce291e9\": container with ID starting with 1c4e9e43a1b476d5b55f96997a042cf63e2023eb90694d2651f319da7ce291e9 not found: ID does not exist" containerID="1c4e9e43a1b476d5b55f96997a042cf63e2023eb90694d2651f319da7ce291e9" Jul 10 19:03:06.984240 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.984203 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c4e9e43a1b476d5b55f96997a042cf63e2023eb90694d2651f319da7ce291e9"} err="failed to get container status \"1c4e9e43a1b476d5b55f96997a042cf63e2023eb90694d2651f319da7ce291e9\": rpc error: code = NotFound desc = could not find container \"1c4e9e43a1b476d5b55f96997a042cf63e2023eb90694d2651f319da7ce291e9\": container with ID starting with 1c4e9e43a1b476d5b55f96997a042cf63e2023eb90694d2651f319da7ce291e9 not found: ID does not exist" Jul 10 19:03:06.984240 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.984220 2571 scope.go:117] "RemoveContainer" containerID="941dc1333d8adb2ec9fe3696ded34f0d7e12879673aab5349d6a810b4690a655" Jul 10 19:03:06.984443 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:06.984427 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"941dc1333d8adb2ec9fe3696ded34f0d7e12879673aab5349d6a810b4690a655\": container with ID starting with 941dc1333d8adb2ec9fe3696ded34f0d7e12879673aab5349d6a810b4690a655 not found: ID does not exist" containerID="941dc1333d8adb2ec9fe3696ded34f0d7e12879673aab5349d6a810b4690a655" Jul 10 19:03:06.984481 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.984449 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"941dc1333d8adb2ec9fe3696ded34f0d7e12879673aab5349d6a810b4690a655"} err="failed to get container status \"941dc1333d8adb2ec9fe3696ded34f0d7e12879673aab5349d6a810b4690a655\": rpc error: code = NotFound desc = could not find container \"941dc1333d8adb2ec9fe3696ded34f0d7e12879673aab5349d6a810b4690a655\": container with ID starting with 941dc1333d8adb2ec9fe3696ded34f0d7e12879673aab5349d6a810b4690a655 not found: ID does not exist" Jul 10 19:03:06.984481 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.984463 2571 scope.go:117] "RemoveContainer" containerID="44a17d61cad01ac43dd6752194bc8e0e258eea55292c5988c18800c9f5977847" Jul 10 19:03:06.984679 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:06.984664 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44a17d61cad01ac43dd6752194bc8e0e258eea55292c5988c18800c9f5977847\": container with ID starting with 44a17d61cad01ac43dd6752194bc8e0e258eea55292c5988c18800c9f5977847 not found: ID does not exist" containerID="44a17d61cad01ac43dd6752194bc8e0e258eea55292c5988c18800c9f5977847" Jul 10 19:03:06.984733 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.984681 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44a17d61cad01ac43dd6752194bc8e0e258eea55292c5988c18800c9f5977847"} err="failed to get container status \"44a17d61cad01ac43dd6752194bc8e0e258eea55292c5988c18800c9f5977847\": rpc error: code = NotFound desc = could not find container \"44a17d61cad01ac43dd6752194bc8e0e258eea55292c5988c18800c9f5977847\": container with ID starting with 44a17d61cad01ac43dd6752194bc8e0e258eea55292c5988c18800c9f5977847 not found: ID does not exist" Jul 10 19:03:06.991436 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.991416 2571 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/627e86ba-0df2-4638-a56c-d9f990f4ff36-metrics-client-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:06.991528 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.991441 2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:06.991528 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.991452 2571 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/627e86ba-0df2-4638-a56c-d9f990f4ff36-config-out\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:06.991528 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.991462 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhp94\" (UniqueName: \"kubernetes.io/projected/627e86ba-0df2-4638-a56c-d9f990f4ff36-kube-api-access-fhp94\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:06.991528 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.991472 2571 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-cluster-tls-config\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:06.991528 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.991480 2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:06.991528 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.991489 2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-main-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:06.991528 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.991497 2571 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/627e86ba-0df2-4638-a56c-d9f990f4ff36-alertmanager-main-db\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:06.991528 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.991508 2571 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-web-config\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:06.991528 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.991521 2571 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/627e86ba-0df2-4638-a56c-d9f990f4ff36-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:06.991528 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.991529 2571 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/627e86ba-0df2-4638-a56c-d9f990f4ff36-tls-assets\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:06.991858 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.991538 2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:06.991858 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.991546 2571 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/627e86ba-0df2-4638-a56c-d9f990f4ff36-config-volume\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:06.992454 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992437 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 19:03:06.992728 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992717 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5501e15d-59e4-4a41-8cb0-77ea136593ca" containerName="registry" Jul 10 19:03:06.992761 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992731 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5501e15d-59e4-4a41-8cb0-77ea136593ca" containerName="registry" Jul 10 19:03:06.992761 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992740 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd20bbb3-4733-4ea4-b5e8-d7fce01202b7" containerName="console" Jul 10 19:03:06.992761 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992746 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd20bbb3-4733-4ea4-b5e8-d7fce01202b7" containerName="console" Jul 10 19:03:06.992761 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992752 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="alertmanager" Jul 10 19:03:06.992761 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992758 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="alertmanager" Jul 10 19:03:06.992895 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992766 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="kube-rbac-proxy-metric" Jul 10 19:03:06.992895 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992772 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="kube-rbac-proxy-metric" Jul 10 19:03:06.992895 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992782 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="kube-rbac-proxy" Jul 10 19:03:06.992895 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992787 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="kube-rbac-proxy" Jul 10 19:03:06.992895 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992795 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="kube-rbac-proxy-web" Jul 10 19:03:06.992895 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992800 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="kube-rbac-proxy-web" Jul 10 19:03:06.992895 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992807 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="prom-label-proxy" Jul 10 19:03:06.992895 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992812 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="prom-label-proxy" Jul 10 19:03:06.992895 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992818 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="init-config-reloader" Jul 10 19:03:06.992895 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992823 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="init-config-reloader" Jul 10 19:03:06.992895 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992837 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="config-reloader" Jul 10 19:03:06.992895 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992843 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="config-reloader" Jul 10 19:03:06.992895 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992849 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e49ee3e6-1da9-49bc-8fc4-5dfcd9250982" containerName="console" Jul 10 19:03:06.992895 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992853 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e49ee3e6-1da9-49bc-8fc4-5dfcd9250982" containerName="console" Jul 10 19:03:06.992895 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992898 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="alertmanager" Jul 10 19:03:06.993317 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992906 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="kube-rbac-proxy" Jul 10 19:03:06.993317 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992911 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd20bbb3-4733-4ea4-b5e8-d7fce01202b7" containerName="console" Jul 10 19:03:06.993317 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992916 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="kube-rbac-proxy-web" Jul 10 19:03:06.993317 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992923 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="kube-rbac-proxy-metric" Jul 10 19:03:06.993317 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992929 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="prom-label-proxy" Jul 10 19:03:06.993317 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992937 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" containerName="config-reloader" Jul 10 19:03:06.993317 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992942 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5501e15d-59e4-4a41-8cb0-77ea136593ca" containerName="registry" Jul 10 19:03:06.993317 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.992947 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e49ee3e6-1da9-49bc-8fc4-5dfcd9250982" containerName="console" Jul 10 19:03:06.996932 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.996917 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:06.999442 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.999424 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 10 19:03:06.999681 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.999645 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 10 19:03:06.999681 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.999659 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 10 19:03:06.999835 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.999707 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 10 19:03:06.999835 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.999723 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-wcmlw\"" Jul 10 19:03:06.999835 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.999725 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 10 19:03:06.999835 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:06.999725 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 10 19:03:07.000064 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.000048 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 10 19:03:07.000130 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.000053 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 10 19:03:07.004857 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.004837 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 10 19:03:07.020315 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.019516 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="627e86ba-0df2-4638-a56c-d9f990f4ff36" path="/var/lib/kubelet/pods/627e86ba-0df2-4638-a56c-d9f990f4ff36/volumes" Jul 10 19:03:07.020315 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.020211 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 19:03:07.092996 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.092971 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-config-volume\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.093160 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.093005 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-web-config\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.093160 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.093045 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.093160 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.093122 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.093160 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.093158 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6798b0dd-81ab-42bd-9ed4-9811a97c0950-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.093300 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.093174 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4hj5\" (UniqueName: \"kubernetes.io/projected/6798b0dd-81ab-42bd-9ed4-9811a97c0950-kube-api-access-x4hj5\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.093300 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.093235 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.093300 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.093266 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6798b0dd-81ab-42bd-9ed4-9811a97c0950-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.093300 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.093287 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.093472 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.093302 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6798b0dd-81ab-42bd-9ed4-9811a97c0950-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.093472 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.093372 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.093472 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.093407 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6798b0dd-81ab-42bd-9ed4-9811a97c0950-config-out\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.093604 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.093472 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6798b0dd-81ab-42bd-9ed4-9811a97c0950-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.194555 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.194476 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.194555 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.194513 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6798b0dd-81ab-42bd-9ed4-9811a97c0950-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.194555 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.194540 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.194798 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.194563 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6798b0dd-81ab-42bd-9ed4-9811a97c0950-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.194798 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.194622 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.194798 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.194650 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6798b0dd-81ab-42bd-9ed4-9811a97c0950-config-out\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.194798 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.194694 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6798b0dd-81ab-42bd-9ed4-9811a97c0950-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.194798 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.194735 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-config-volume\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.195515 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.194952 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-web-config\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.195515 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.195023 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.195515 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.195059 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.195515 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.195087 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6798b0dd-81ab-42bd-9ed4-9811a97c0950-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.195515 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.195141 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4hj5\" (UniqueName: \"kubernetes.io/projected/6798b0dd-81ab-42bd-9ed4-9811a97c0950-kube-api-access-x4hj5\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.195766 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.195527 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6798b0dd-81ab-42bd-9ed4-9811a97c0950-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.195766 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.195632 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6798b0dd-81ab-42bd-9ed4-9811a97c0950-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.196341 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.196314 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6798b0dd-81ab-42bd-9ed4-9811a97c0950-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.197450 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.197424 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6798b0dd-81ab-42bd-9ed4-9811a97c0950-config-out\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.197612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.197591 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.197720 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.197686 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6798b0dd-81ab-42bd-9ed4-9811a97c0950-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.197769 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.197732 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.198485 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.198459 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-config-volume\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.198585 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.198500 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.198585 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.198552 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-web-config\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.198768 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.198750 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.199449 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.199432 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6798b0dd-81ab-42bd-9ed4-9811a97c0950-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.202983 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.202965 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4hj5\" (UniqueName: \"kubernetes.io/projected/6798b0dd-81ab-42bd-9ed4-9811a97c0950-kube-api-access-x4hj5\") pod \"alertmanager-main-0\" (UID: \"6798b0dd-81ab-42bd-9ed4-9811a97c0950\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.306202 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.306159 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 19:03:07.441220 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.441195 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 19:03:07.442902 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:03:07.442877 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6798b0dd_81ab_42bd_9ed4_9811a97c0950.slice/crio-86982486e9e9630121428c7681dba562ce74550715a6a5559801a8b1ee201de7 WatchSource:0}: Error finding container 86982486e9e9630121428c7681dba562ce74550715a6a5559801a8b1ee201de7: Status 404 returned error can't find the container with id 86982486e9e9630121428c7681dba562ce74550715a6a5559801a8b1ee201de7 Jul 10 19:03:07.937741 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.937709 2571 generic.go:356] "Generic (PLEG): container finished" podID="6798b0dd-81ab-42bd-9ed4-9811a97c0950" containerID="acc4065521e478fb22447b173eadf1fec234e6d66fb3a81aa84b485102e97783" exitCode=0 Jul 10 19:03:07.938189 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.937801 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6798b0dd-81ab-42bd-9ed4-9811a97c0950","Type":"ContainerDied","Data":"acc4065521e478fb22447b173eadf1fec234e6d66fb3a81aa84b485102e97783"} Jul 10 19:03:07.938189 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:07.937861 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6798b0dd-81ab-42bd-9ed4-9811a97c0950","Type":"ContainerStarted","Data":"86982486e9e9630121428c7681dba562ce74550715a6a5559801a8b1ee201de7"} Jul 10 19:03:08.944757 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:08.944724 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6798b0dd-81ab-42bd-9ed4-9811a97c0950","Type":"ContainerStarted","Data":"41af9f3623d15537a2f62778569750937f717aa8433603ba77d179ede775bf70"} Jul 10 19:03:08.945138 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:08.944787 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6798b0dd-81ab-42bd-9ed4-9811a97c0950","Type":"ContainerStarted","Data":"f24796d6bd2bd4c0fed58847fb58b5d05aa60b6ede85c4ad4dc6b35fef30baec"} Jul 10 19:03:08.945138 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:08.944811 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6798b0dd-81ab-42bd-9ed4-9811a97c0950","Type":"ContainerStarted","Data":"47126fbb4242634ba5833bc87d50afafc78080874c7463257f0ec268767180cb"} Jul 10 19:03:08.945138 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:08.944835 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6798b0dd-81ab-42bd-9ed4-9811a97c0950","Type":"ContainerStarted","Data":"9c9032d280b285004ae343835e1bd04b216936aa3a8bac5e6e24e97374ea809b"} Jul 10 19:03:08.945138 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:08.944859 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6798b0dd-81ab-42bd-9ed4-9811a97c0950","Type":"ContainerStarted","Data":"a67a6205150e31951e37a6ae5898710efcff882e45451d1f31c6392d0275bae0"} Jul 10 19:03:08.945138 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:08.944883 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6798b0dd-81ab-42bd-9ed4-9811a97c0950","Type":"ContainerStarted","Data":"98b2b32eed831d15f9a9a9e8482aa923b4fe891b29a5c862c4de473e49a01185"} Jul 10 19:03:08.975917 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:08.975864 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.9566962 podStartE2EDuration="2.975849254s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.153054ms" isStatefulPod=false podCreationTimestamp="2026-07-10 19:03:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:03:08.973200056 +0000 UTC m=+170.509542488" watchObservedRunningTime="2026-07-10 19:03:08.975849254 +0000 UTC m=+170.512191686" Jul 10 19:03:09.817448 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:09.817406 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 19:03:09.818483 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:09.818422 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="prometheus" containerID="cri-o://89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88" gracePeriod=600 Jul 10 19:03:09.818483 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:09.818462 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="kube-rbac-proxy-web" containerID="cri-o://590ebaeb087ffd9986d021414fe3237d67beedcfca440a750a83fe86adc9520b" gracePeriod=600 Jul 10 19:03:09.818661 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:09.818462 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="kube-rbac-proxy" containerID="cri-o://a5e325338aca107a64c466307f3288236572dc9915fe7cd1438d845b263c073a" gracePeriod=600 Jul 10 19:03:09.818661 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:09.818474 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="kube-rbac-proxy-thanos" containerID="cri-o://3795300ec5cc9539dfe7443d97b1653925dfb2b297afb3870066ffdfaba3dcdb" gracePeriod=600 Jul 10 19:03:09.818661 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:09.818492 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="thanos-sidecar" containerID="cri-o://f9b843a80e524772a5b443bfef89b1f0864ffb123155a3f0442d26daff68c04e" gracePeriod=600 Jul 10 19:03:09.818661 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:09.818597 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="config-reloader" containerID="cri-o://cd1696abe9f32e82f78097fa8f10577f2ec84ce55bcdae3813b4f853d7526f13" gracePeriod=600 Jul 10 19:03:09.953133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:09.953084 2571 generic.go:356] "Generic (PLEG): container finished" podID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerID="590ebaeb087ffd9986d021414fe3237d67beedcfca440a750a83fe86adc9520b" exitCode=0 Jul 10 19:03:09.953133 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:09.953128 2571 generic.go:356] "Generic (PLEG): container finished" podID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerID="f9b843a80e524772a5b443bfef89b1f0864ffb123155a3f0442d26daff68c04e" exitCode=0 Jul 10 19:03:09.953552 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:09.953141 2571 generic.go:356] "Generic (PLEG): container finished" podID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerID="cd1696abe9f32e82f78097fa8f10577f2ec84ce55bcdae3813b4f853d7526f13" exitCode=0 Jul 10 19:03:09.953552 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:09.953137 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15c278e5-f3a7-438a-8f98-d0acbadf2484","Type":"ContainerDied","Data":"590ebaeb087ffd9986d021414fe3237d67beedcfca440a750a83fe86adc9520b"} Jul 10 19:03:09.953552 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:09.953222 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15c278e5-f3a7-438a-8f98-d0acbadf2484","Type":"ContainerDied","Data":"f9b843a80e524772a5b443bfef89b1f0864ffb123155a3f0442d26daff68c04e"} Jul 10 19:03:09.953552 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:09.953272 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15c278e5-f3a7-438a-8f98-d0acbadf2484","Type":"ContainerDied","Data":"cd1696abe9f32e82f78097fa8f10577f2ec84ce55bcdae3813b4f853d7526f13"} Jul 10 19:03:10.962086 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:10.962050 2571 generic.go:356] "Generic (PLEG): container finished" podID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerID="3795300ec5cc9539dfe7443d97b1653925dfb2b297afb3870066ffdfaba3dcdb" exitCode=0 Jul 10 19:03:10.962086 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:10.962078 2571 generic.go:356] "Generic (PLEG): container finished" podID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerID="a5e325338aca107a64c466307f3288236572dc9915fe7cd1438d845b263c073a" exitCode=0 Jul 10 19:03:10.962406 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:10.962132 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15c278e5-f3a7-438a-8f98-d0acbadf2484","Type":"ContainerDied","Data":"3795300ec5cc9539dfe7443d97b1653925dfb2b297afb3870066ffdfaba3dcdb"} Jul 10 19:03:10.962406 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:10.962210 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15c278e5-f3a7-438a-8f98-d0acbadf2484","Type":"ContainerDied","Data":"a5e325338aca107a64c466307f3288236572dc9915fe7cd1438d845b263c073a"} Jul 10 19:03:11.869923 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:11.869874 2571 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"] Jul 10 19:03:11.870932 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:11.870891 2571 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"] Jul 10 19:03:11.871805 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:11.871774 2571 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"] Jul 10 19:03:11.871881 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:11.871816 2571 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-0" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="prometheus" probeResult="unknown" Jul 10 19:03:13.729032 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:13.729002 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f4c68b74f-vprxv"] Jul 10 19:03:16.869004 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:16.868959 2571 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88 is running failed: container process not found" containerID="89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"] Jul 10 19:03:16.869399 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:16.869345 2571 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88 is running failed: container process not found" containerID="89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"] Jul 10 19:03:16.869668 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:16.869644 2571 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88 is running failed: container process not found" containerID="89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"] Jul 10 19:03:16.869732 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:16.869691 2571 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88 is running failed: container process not found" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-0" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="prometheus" probeResult="unknown" Jul 10 19:03:16.975578 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:16.975557 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:16.990811 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:16.990786 2571 generic.go:356] "Generic (PLEG): container finished" podID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerID="89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88" exitCode=0 Jul 10 19:03:16.990919 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:16.990836 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15c278e5-f3a7-438a-8f98-d0acbadf2484","Type":"ContainerDied","Data":"89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88"} Jul 10 19:03:16.990919 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:16.990862 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:16.990919 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:16.990897 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15c278e5-f3a7-438a-8f98-d0acbadf2484","Type":"ContainerDied","Data":"ed8804f251bf34f9b8bcd94df0cb24902dedc364f5c72fac93029c801d11aa64"} Jul 10 19:03:16.991083 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:16.990931 2571 scope.go:117] "RemoveContainer" containerID="3795300ec5cc9539dfe7443d97b1653925dfb2b297afb3870066ffdfaba3dcdb" Jul 10 19:03:16.998335 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:16.998317 2571 scope.go:117] "RemoveContainer" containerID="a5e325338aca107a64c466307f3288236572dc9915fe7cd1438d845b263c073a" Jul 10 19:03:17.004831 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.004815 2571 scope.go:117] "RemoveContainer" containerID="590ebaeb087ffd9986d021414fe3237d67beedcfca440a750a83fe86adc9520b" Jul 10 19:03:17.012880 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.012861 2571 scope.go:117] "RemoveContainer" containerID="f9b843a80e524772a5b443bfef89b1f0864ffb123155a3f0442d26daff68c04e" Jul 10 19:03:17.021214 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.021192 2571 scope.go:117] "RemoveContainer" containerID="cd1696abe9f32e82f78097fa8f10577f2ec84ce55bcdae3813b4f853d7526f13" Jul 10 19:03:17.030461 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.030445 2571 scope.go:117] "RemoveContainer" containerID="89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88" Jul 10 19:03:17.037923 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.037898 2571 scope.go:117] "RemoveContainer" containerID="8d7cfd7d05ec94380d7e4a72ff312d4735f7895781bce64df7c695b755004e19" Jul 10 19:03:17.044897 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.044881 2571 scope.go:117] "RemoveContainer" containerID="3795300ec5cc9539dfe7443d97b1653925dfb2b297afb3870066ffdfaba3dcdb" Jul 10 19:03:17.045171 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:17.045149 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3795300ec5cc9539dfe7443d97b1653925dfb2b297afb3870066ffdfaba3dcdb\": container with ID starting with 3795300ec5cc9539dfe7443d97b1653925dfb2b297afb3870066ffdfaba3dcdb not found: ID does not exist" containerID="3795300ec5cc9539dfe7443d97b1653925dfb2b297afb3870066ffdfaba3dcdb" Jul 10 19:03:17.045238 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.045179 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3795300ec5cc9539dfe7443d97b1653925dfb2b297afb3870066ffdfaba3dcdb"} err="failed to get container status \"3795300ec5cc9539dfe7443d97b1653925dfb2b297afb3870066ffdfaba3dcdb\": rpc error: code = NotFound desc = could not find container \"3795300ec5cc9539dfe7443d97b1653925dfb2b297afb3870066ffdfaba3dcdb\": container with ID starting with 3795300ec5cc9539dfe7443d97b1653925dfb2b297afb3870066ffdfaba3dcdb not found: ID does not exist" Jul 10 19:03:17.045238 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.045197 2571 scope.go:117] "RemoveContainer" containerID="a5e325338aca107a64c466307f3288236572dc9915fe7cd1438d845b263c073a" Jul 10 19:03:17.045435 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:17.045416 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5e325338aca107a64c466307f3288236572dc9915fe7cd1438d845b263c073a\": container with ID starting with a5e325338aca107a64c466307f3288236572dc9915fe7cd1438d845b263c073a not found: ID does not exist" containerID="a5e325338aca107a64c466307f3288236572dc9915fe7cd1438d845b263c073a" Jul 10 19:03:17.045540 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.045436 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5e325338aca107a64c466307f3288236572dc9915fe7cd1438d845b263c073a"} err="failed to get container status \"a5e325338aca107a64c466307f3288236572dc9915fe7cd1438d845b263c073a\": rpc error: code = NotFound desc = could not find container \"a5e325338aca107a64c466307f3288236572dc9915fe7cd1438d845b263c073a\": container with ID starting with a5e325338aca107a64c466307f3288236572dc9915fe7cd1438d845b263c073a not found: ID does not exist" Jul 10 19:03:17.045540 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.045451 2571 scope.go:117] "RemoveContainer" containerID="590ebaeb087ffd9986d021414fe3237d67beedcfca440a750a83fe86adc9520b" Jul 10 19:03:17.045694 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:17.045679 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"590ebaeb087ffd9986d021414fe3237d67beedcfca440a750a83fe86adc9520b\": container with ID starting with 590ebaeb087ffd9986d021414fe3237d67beedcfca440a750a83fe86adc9520b not found: ID does not exist" containerID="590ebaeb087ffd9986d021414fe3237d67beedcfca440a750a83fe86adc9520b" Jul 10 19:03:17.045729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.045698 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"590ebaeb087ffd9986d021414fe3237d67beedcfca440a750a83fe86adc9520b"} err="failed to get container status \"590ebaeb087ffd9986d021414fe3237d67beedcfca440a750a83fe86adc9520b\": rpc error: code = NotFound desc = could not find container \"590ebaeb087ffd9986d021414fe3237d67beedcfca440a750a83fe86adc9520b\": container with ID starting with 590ebaeb087ffd9986d021414fe3237d67beedcfca440a750a83fe86adc9520b not found: ID does not exist" Jul 10 19:03:17.045729 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.045716 2571 scope.go:117] "RemoveContainer" containerID="f9b843a80e524772a5b443bfef89b1f0864ffb123155a3f0442d26daff68c04e" Jul 10 19:03:17.045949 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:17.045934 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9b843a80e524772a5b443bfef89b1f0864ffb123155a3f0442d26daff68c04e\": container with ID starting with f9b843a80e524772a5b443bfef89b1f0864ffb123155a3f0442d26daff68c04e not found: ID does not exist" containerID="f9b843a80e524772a5b443bfef89b1f0864ffb123155a3f0442d26daff68c04e" Jul 10 19:03:17.046002 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.045952 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9b843a80e524772a5b443bfef89b1f0864ffb123155a3f0442d26daff68c04e"} err="failed to get container status \"f9b843a80e524772a5b443bfef89b1f0864ffb123155a3f0442d26daff68c04e\": rpc error: code = NotFound desc = could not find container \"f9b843a80e524772a5b443bfef89b1f0864ffb123155a3f0442d26daff68c04e\": container with ID starting with f9b843a80e524772a5b443bfef89b1f0864ffb123155a3f0442d26daff68c04e not found: ID does not exist" Jul 10 19:03:17.046002 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.045969 2571 scope.go:117] "RemoveContainer" containerID="cd1696abe9f32e82f78097fa8f10577f2ec84ce55bcdae3813b4f853d7526f13" Jul 10 19:03:17.046232 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:17.046216 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd1696abe9f32e82f78097fa8f10577f2ec84ce55bcdae3813b4f853d7526f13\": container with ID starting with cd1696abe9f32e82f78097fa8f10577f2ec84ce55bcdae3813b4f853d7526f13 not found: ID does not exist" containerID="cd1696abe9f32e82f78097fa8f10577f2ec84ce55bcdae3813b4f853d7526f13" Jul 10 19:03:17.046289 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.046234 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd1696abe9f32e82f78097fa8f10577f2ec84ce55bcdae3813b4f853d7526f13"} err="failed to get container status \"cd1696abe9f32e82f78097fa8f10577f2ec84ce55bcdae3813b4f853d7526f13\": rpc error: code = NotFound desc = could not find container \"cd1696abe9f32e82f78097fa8f10577f2ec84ce55bcdae3813b4f853d7526f13\": container with ID starting with cd1696abe9f32e82f78097fa8f10577f2ec84ce55bcdae3813b4f853d7526f13 not found: ID does not exist" Jul 10 19:03:17.046289 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.046258 2571 scope.go:117] "RemoveContainer" containerID="89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88" Jul 10 19:03:17.046439 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:17.046424 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88\": container with ID starting with 89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88 not found: ID does not exist" containerID="89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88" Jul 10 19:03:17.046503 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.046443 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88"} err="failed to get container status \"89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88\": rpc error: code = NotFound desc = could not find container \"89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88\": container with ID starting with 89dbf4504f861f5e99d1780a641006c6d946d1274cec76c39670cc26148fda88 not found: ID does not exist" Jul 10 19:03:17.046503 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.046457 2571 scope.go:117] "RemoveContainer" containerID="8d7cfd7d05ec94380d7e4a72ff312d4735f7895781bce64df7c695b755004e19" Jul 10 19:03:17.046722 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:17.046699 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d7cfd7d05ec94380d7e4a72ff312d4735f7895781bce64df7c695b755004e19\": container with ID starting with 8d7cfd7d05ec94380d7e4a72ff312d4735f7895781bce64df7c695b755004e19 not found: ID does not exist" containerID="8d7cfd7d05ec94380d7e4a72ff312d4735f7895781bce64df7c695b755004e19" Jul 10 19:03:17.046790 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.046728 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7cfd7d05ec94380d7e4a72ff312d4735f7895781bce64df7c695b755004e19"} err="failed to get container status \"8d7cfd7d05ec94380d7e4a72ff312d4735f7895781bce64df7c695b755004e19\": rpc error: code = NotFound desc = could not find container \"8d7cfd7d05ec94380d7e4a72ff312d4735f7895781bce64df7c695b755004e19\": container with ID starting with 8d7cfd7d05ec94380d7e4a72ff312d4735f7895781bce64df7c695b755004e19 not found: ID does not exist" Jul 10 19:03:17.084243 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.084192 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-configmap-kubelet-serving-ca-bundle\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.084243 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.084224 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.084354 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.084251 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-prometheus-k8s-tls\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.084354 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.084278 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-db\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.084438 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.084351 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-configmap-metrics-client-ca\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.084438 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.084377 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/15c278e5-f3a7-438a-8f98-d0acbadf2484-tls-assets\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.084438 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.084402 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-trusted-ca-bundle\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.084438 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.084430 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-thanos-prometheus-http-client-file\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.084634 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.084610 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:03:17.084695 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.084663 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.084788 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.084707 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnrl9\" (UniqueName: \"kubernetes.io/projected/15c278e5-f3a7-438a-8f98-d0acbadf2484-kube-api-access-nnrl9\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.084788 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.084749 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:03:17.084788 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.084768 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-config\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.084955 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.084810 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-rulefiles-1\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.085058 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.085038 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-configmap-serving-certs-ca-bundle\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.085135 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.085075 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-rulefiles-2\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.085135 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.085130 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/15c278e5-f3a7-438a-8f98-d0acbadf2484-config-out\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.085323 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.085303 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-web-config\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.085520 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.085437 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:03:17.085520 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.085477 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:03:17.086725 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.085668 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-grpc-tls\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.086725 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.085736 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-metrics-client-certs\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.086725 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.085794 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-rulefiles-0\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.086725 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.085828 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-kube-rbac-proxy\") pod \"15c278e5-f3a7-438a-8f98-d0acbadf2484\" (UID: \"15c278e5-f3a7-438a-8f98-d0acbadf2484\") " Jul 10 19:03:17.086725 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.085967 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:03:17.086725 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.086263 2571 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-configmap-metrics-client-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.086725 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.086288 2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.086725 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.086295 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:03:17.086725 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.086307 2571 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.086725 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.086344 2571 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.086725 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.086362 2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-db\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.086725 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.086684 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:03:17.087755 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.087726 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:03:17.088561 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.088532 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:03:17.088766 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.088743 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:03:17.088981 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.088953 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:03:17.089363 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.089263 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:03:17.089363 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.089298 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15c278e5-f3a7-438a-8f98-d0acbadf2484-kube-api-access-nnrl9" (OuterVolumeSpecName: "kube-api-access-nnrl9") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "kube-api-access-nnrl9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:03:17.089363 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.089322 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:03:17.089363 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.089337 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15c278e5-f3a7-438a-8f98-d0acbadf2484-config-out" (OuterVolumeSpecName: "config-out") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:03:17.089538 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.089408 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15c278e5-f3a7-438a-8f98-d0acbadf2484-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:03:17.089998 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.089974 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:03:17.090059 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.090012 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:03:17.090826 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.090801 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-config" (OuterVolumeSpecName: "config") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:03:17.099477 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.099453 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-web-config" (OuterVolumeSpecName: "web-config") pod "15c278e5-f3a7-438a-8f98-d0acbadf2484" (UID: "15c278e5-f3a7-438a-8f98-d0acbadf2484"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:03:17.187156 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.187123 2571 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-config\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.187156 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.187155 2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.187156 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.187165 2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.187306 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.187176 2571 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/15c278e5-f3a7-438a-8f98-d0acbadf2484-config-out\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.187306 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.187184 2571 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-web-config\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.187306 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.187191 2571 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-grpc-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.187306 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.187200 2571 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-metrics-client-certs\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.187306 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.187208 2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/15c278e5-f3a7-438a-8f98-d0acbadf2484-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.187306 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.187217 2571 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-kube-rbac-proxy\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.187306 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.187225 2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.187306 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.187234 2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.187306 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.187242 2571 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/15c278e5-f3a7-438a-8f98-d0acbadf2484-tls-assets\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.187306 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.187250 2571 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.187306 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.187258 2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/15c278e5-f3a7-438a-8f98-d0acbadf2484-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.187306 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.187266 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnrl9\" (UniqueName: \"kubernetes.io/projected/15c278e5-f3a7-438a-8f98-d0acbadf2484-kube-api-access-nnrl9\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:17.316069 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.316045 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 19:03:17.320244 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.320224 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 19:03:17.355118 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355052 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 19:03:17.355371 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355357 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="thanos-sidecar" Jul 10 19:03:17.355414 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355373 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="thanos-sidecar" Jul 10 19:03:17.355414 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355385 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="kube-rbac-proxy-web" Jul 10 19:03:17.355414 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355391 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="kube-rbac-proxy-web" Jul 10 19:03:17.355414 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355399 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="kube-rbac-proxy" Jul 10 19:03:17.355414 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355405 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="kube-rbac-proxy" Jul 10 19:03:17.355562 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355418 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="config-reloader" Jul 10 19:03:17.355562 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355424 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="config-reloader" Jul 10 19:03:17.355562 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355432 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="init-config-reloader" Jul 10 19:03:17.355562 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355437 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="init-config-reloader" Jul 10 19:03:17.355562 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355448 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="prometheus" Jul 10 19:03:17.355562 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355454 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="prometheus" Jul 10 19:03:17.355562 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355459 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="kube-rbac-proxy-thanos" Jul 10 19:03:17.355562 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355464 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="kube-rbac-proxy-thanos" Jul 10 19:03:17.355562 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355514 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="thanos-sidecar" Jul 10 19:03:17.355562 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355524 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="kube-rbac-proxy-thanos" Jul 10 19:03:17.355562 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355531 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="prometheus" Jul 10 19:03:17.355562 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355538 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="config-reloader" Jul 10 19:03:17.355562 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355543 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="kube-rbac-proxy-web" Jul 10 19:03:17.355562 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.355550 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" containerName="kube-rbac-proxy" Jul 10 19:03:17.358915 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.358898 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.363505 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.363216 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 10 19:03:17.363505 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.363253 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 10 19:03:17.363505 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.363348 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-m9wkx\"" Jul 10 19:03:17.363505 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.363362 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 10 19:03:17.363505 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.363479 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 10 19:03:17.363505 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.363484 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 10 19:03:17.363837 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.363611 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 10 19:03:17.363837 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.363811 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 10 19:03:17.363942 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.363849 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 10 19:03:17.363942 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.363910 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 10 19:03:17.364118 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.364084 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b13s5uq4qjc4e\"" Jul 10 19:03:17.364210 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.364190 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 10 19:03:17.364309 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.364191 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 10 19:03:17.364369 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.364301 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 10 19:03:17.365697 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.365678 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 10 19:03:17.368388 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.368370 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 10 19:03:17.374710 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.374689 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 19:03:17.389326 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389306 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-config\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389408 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389336 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389408 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389350 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389408 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389392 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389507 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389433 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389507 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389474 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389574 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389519 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389574 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389537 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-web-config\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389574 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389562 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389654 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389592 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389654 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389617 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knmp7\" (UniqueName: \"kubernetes.io/projected/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-kube-api-access-knmp7\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389660 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389717 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389700 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389777 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389728 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389777 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389746 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389777 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389761 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389870 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389781 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389870 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389831 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-config-out\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389929 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389865 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.389929 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.389889 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.490938 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.490914 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-config-out\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491039 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.490943 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491039 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.490960 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491039 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.490982 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-config\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491039 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.491004 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491277 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.491126 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491277 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.491180 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491277 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.491208 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491277 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.491234 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491277 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.491272 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491512 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.491293 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-web-config\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491512 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.491333 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491512 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.491371 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491512 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.491395 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knmp7\" (UniqueName: \"kubernetes.io/projected/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-kube-api-access-knmp7\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491512 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.491409 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491512 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.491430 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491512 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.491479 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491512 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.491504 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491892 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.491532 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491892 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.491555 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491892 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.491585 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.491892 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.491868 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.492573 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.492545 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.493259 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.493223 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.493828 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.493802 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-config-out\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.494155 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.494128 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.494282 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.494260 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.494282 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.494270 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.494541 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.494514 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.494851 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.494827 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.495452 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.494955 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.495452 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.495382 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-config\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.495819 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.495794 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.495895 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.495858 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.496256 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.496229 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.496389 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.496370 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-web-config\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.496726 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.496704 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.496799 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.496783 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.496885 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.496868 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.503025 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.503009 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knmp7\" (UniqueName: \"kubernetes.io/projected/164024dc-6cbc-4dce-b9f9-f3b4ea73ac56-kube-api-access-knmp7\") pod \"prometheus-k8s-0\" (UID: \"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.670289 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.670230 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:17.811927 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.811904 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 19:03:17.813507 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:03:17.813467 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod164024dc_6cbc_4dce_b9f9_f3b4ea73ac56.slice/crio-0da77f723f755dda4a763e2f67a01817316cf9f35b2ab0f8839c8917d82128e3 WatchSource:0}: Error finding container 0da77f723f755dda4a763e2f67a01817316cf9f35b2ab0f8839c8917d82128e3: Status 404 returned error can't find the container with id 0da77f723f755dda4a763e2f67a01817316cf9f35b2ab0f8839c8917d82128e3 Jul 10 19:03:17.997920 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.997856 2571 generic.go:356] "Generic (PLEG): container finished" podID="164024dc-6cbc-4dce-b9f9-f3b4ea73ac56" containerID="fe37fc0939f306d7c0a7de9e8f29ab3fe3e5f0922e23b2108278268214278049" exitCode=0 Jul 10 19:03:17.998231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.997918 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56","Type":"ContainerDied","Data":"fe37fc0939f306d7c0a7de9e8f29ab3fe3e5f0922e23b2108278268214278049"} Jul 10 19:03:17.998231 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:17.997974 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56","Type":"ContainerStarted","Data":"0da77f723f755dda4a763e2f67a01817316cf9f35b2ab0f8839c8917d82128e3"} Jul 10 19:03:19.005362 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:19.005324 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56","Type":"ContainerStarted","Data":"b58c33e215b9d60177ef110f1232432b72955aada13708952b09b138c07f8f96"} Jul 10 19:03:19.005831 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:19.005425 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56","Type":"ContainerStarted","Data":"decf73cfc33749d9eebb4625965b1b4ba6ba92e2631846ecd6313eda40df6af6"} Jul 10 19:03:19.005831 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:19.005466 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56","Type":"ContainerStarted","Data":"cf8070cb32a98477cecf57c45a703c74726fc57f9c19c1956f303b2c541c74ab"} Jul 10 19:03:19.005831 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:19.005511 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56","Type":"ContainerStarted","Data":"4ee4b52bbec3832fa5478e2cd50b529f1395894c1668c7e684c467ec29eab97e"} Jul 10 19:03:19.005831 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:19.005545 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56","Type":"ContainerStarted","Data":"7b5445c0ac77d4fe051612bd86a33357ca696d061b91e8073abce14b72852ac3"} Jul 10 19:03:19.005831 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:19.005609 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"164024dc-6cbc-4dce-b9f9-f3b4ea73ac56","Type":"ContainerStarted","Data":"3f81ddc15dcd805ed0000c4ace923d38d2e2b2c72afcdaa584fc633572ae7101"} Jul 10 19:03:19.017388 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:19.017359 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15c278e5-f3a7-438a-8f98-d0acbadf2484" path="/var/lib/kubelet/pods/15c278e5-f3a7-438a-8f98-d0acbadf2484/volumes" Jul 10 19:03:19.040214 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:19.040167 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.021020306 podStartE2EDuration="2.040149433s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.129127ms" isStatefulPod=false podCreationTimestamp="2026-07-10 19:03:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:03:19.03743007 +0000 UTC m=+180.573772532" watchObservedRunningTime="2026-07-10 19:03:19.040149433 +0000 UTC m=+180.576491865" Jul 10 19:03:22.670984 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:22.670953 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:03:38.749772 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:38.749725 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-f4c68b74f-vprxv" podUID="99f22134-a60c-4ed5-9d46-8e0c15815d38" containerName="console" containerID="cri-o://67b3681b3b567d5e0d1f2ee3e058908a449d2dee870d26742ad44969d9cc5d09" gracePeriod=15 Jul 10 19:03:39.021563 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.021536 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:03:39.085259 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.085224 2571 generic.go:356] "Generic (PLEG): container finished" podID="99f22134-a60c-4ed5-9d46-8e0c15815d38" containerID="67b3681b3b567d5e0d1f2ee3e058908a449d2dee870d26742ad44969d9cc5d09" exitCode=0 Jul 10 19:03:39.085391 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.085270 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f4c68b74f-vprxv" Jul 10 19:03:39.085391 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.085304 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f4c68b74f-vprxv" event={"ID":"99f22134-a60c-4ed5-9d46-8e0c15815d38","Type":"ContainerDied","Data":"67b3681b3b567d5e0d1f2ee3e058908a449d2dee870d26742ad44969d9cc5d09"} Jul 10 19:03:39.085391 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.085354 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f4c68b74f-vprxv" event={"ID":"99f22134-a60c-4ed5-9d46-8e0c15815d38","Type":"ContainerDied","Data":"13800a0120c35d1fa2adf21be339ea87721f27d6286a4484b1a14e3daa211eb8"} Jul 10 19:03:39.085391 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.085377 2571 scope.go:117] "RemoveContainer" containerID="67b3681b3b567d5e0d1f2ee3e058908a449d2dee870d26742ad44969d9cc5d09" Jul 10 19:03:39.093118 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.093091 2571 scope.go:117] "RemoveContainer" containerID="67b3681b3b567d5e0d1f2ee3e058908a449d2dee870d26742ad44969d9cc5d09" Jul 10 19:03:39.093369 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:03:39.093351 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67b3681b3b567d5e0d1f2ee3e058908a449d2dee870d26742ad44969d9cc5d09\": container with ID starting with 67b3681b3b567d5e0d1f2ee3e058908a449d2dee870d26742ad44969d9cc5d09 not found: ID does not exist" containerID="67b3681b3b567d5e0d1f2ee3e058908a449d2dee870d26742ad44969d9cc5d09" Jul 10 19:03:39.093411 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.093376 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67b3681b3b567d5e0d1f2ee3e058908a449d2dee870d26742ad44969d9cc5d09"} err="failed to get container status \"67b3681b3b567d5e0d1f2ee3e058908a449d2dee870d26742ad44969d9cc5d09\": rpc error: code = NotFound desc = could not find container \"67b3681b3b567d5e0d1f2ee3e058908a449d2dee870d26742ad44969d9cc5d09\": container with ID starting with 67b3681b3b567d5e0d1f2ee3e058908a449d2dee870d26742ad44969d9cc5d09 not found: ID does not exist" Jul 10 19:03:39.187197 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.187176 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99f22134-a60c-4ed5-9d46-8e0c15815d38-console-serving-cert\") pod \"99f22134-a60c-4ed5-9d46-8e0c15815d38\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " Jul 10 19:03:39.187318 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.187221 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-trusted-ca-bundle\") pod \"99f22134-a60c-4ed5-9d46-8e0c15815d38\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " Jul 10 19:03:39.187318 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.187245 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-console-config\") pod \"99f22134-a60c-4ed5-9d46-8e0c15815d38\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " Jul 10 19:03:39.187318 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.187273 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-service-ca\") pod \"99f22134-a60c-4ed5-9d46-8e0c15815d38\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " Jul 10 19:03:39.187466 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.187343 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-oauth-serving-cert\") pod \"99f22134-a60c-4ed5-9d46-8e0c15815d38\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " Jul 10 19:03:39.187466 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.187388 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgx5t\" (UniqueName: \"kubernetes.io/projected/99f22134-a60c-4ed5-9d46-8e0c15815d38-kube-api-access-hgx5t\") pod \"99f22134-a60c-4ed5-9d46-8e0c15815d38\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " Jul 10 19:03:39.187466 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.187440 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99f22134-a60c-4ed5-9d46-8e0c15815d38-console-oauth-config\") pod \"99f22134-a60c-4ed5-9d46-8e0c15815d38\" (UID: \"99f22134-a60c-4ed5-9d46-8e0c15815d38\") " Jul 10 19:03:39.187692 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.187660 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "99f22134-a60c-4ed5-9d46-8e0c15815d38" (UID: "99f22134-a60c-4ed5-9d46-8e0c15815d38"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:03:39.187791 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.187764 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "99f22134-a60c-4ed5-9d46-8e0c15815d38" (UID: "99f22134-a60c-4ed5-9d46-8e0c15815d38"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:03:39.187791 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.187770 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-service-ca" (OuterVolumeSpecName: "service-ca") pod "99f22134-a60c-4ed5-9d46-8e0c15815d38" (UID: "99f22134-a60c-4ed5-9d46-8e0c15815d38"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:03:39.187891 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.187868 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-console-config" (OuterVolumeSpecName: "console-config") pod "99f22134-a60c-4ed5-9d46-8e0c15815d38" (UID: "99f22134-a60c-4ed5-9d46-8e0c15815d38"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:03:39.189312 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.189287 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f22134-a60c-4ed5-9d46-8e0c15815d38-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "99f22134-a60c-4ed5-9d46-8e0c15815d38" (UID: "99f22134-a60c-4ed5-9d46-8e0c15815d38"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:03:39.189504 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.189472 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99f22134-a60c-4ed5-9d46-8e0c15815d38-kube-api-access-hgx5t" (OuterVolumeSpecName: "kube-api-access-hgx5t") pod "99f22134-a60c-4ed5-9d46-8e0c15815d38" (UID: "99f22134-a60c-4ed5-9d46-8e0c15815d38"). InnerVolumeSpecName "kube-api-access-hgx5t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:03:39.189551 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.189537 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f22134-a60c-4ed5-9d46-8e0c15815d38-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "99f22134-a60c-4ed5-9d46-8e0c15815d38" (UID: "99f22134-a60c-4ed5-9d46-8e0c15815d38"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:03:39.288680 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.288659 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99f22134-a60c-4ed5-9d46-8e0c15815d38-console-serving-cert\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:39.288680 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.288679 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-trusted-ca-bundle\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:39.288798 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.288688 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-console-config\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:39.288798 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.288698 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:39.288798 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.288706 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99f22134-a60c-4ed5-9d46-8e0c15815d38-oauth-serving-cert\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:39.288798 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.288714 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgx5t\" (UniqueName: \"kubernetes.io/projected/99f22134-a60c-4ed5-9d46-8e0c15815d38-kube-api-access-hgx5t\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:39.288798 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.288723 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99f22134-a60c-4ed5-9d46-8e0c15815d38-console-oauth-config\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:03:39.407836 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.407614 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f4c68b74f-vprxv"] Jul 10 19:03:39.409482 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:39.409459 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f4c68b74f-vprxv"] Jul 10 19:03:41.017758 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:03:41.017725 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99f22134-a60c-4ed5-9d46-8e0c15815d38" path="/var/lib/kubelet/pods/99f22134-a60c-4ed5-9d46-8e0c15815d38/volumes" Jul 10 19:04:17.670761 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:17.670724 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:04:17.686565 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:17.686541 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:04:18.255682 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:18.255655 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 19:04:22.931024 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:22.930988 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lpxzw"] Jul 10 19:04:22.931507 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:22.931367 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99f22134-a60c-4ed5-9d46-8e0c15815d38" containerName="console" Jul 10 19:04:22.931507 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:22.931380 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="99f22134-a60c-4ed5-9d46-8e0c15815d38" containerName="console" Jul 10 19:04:22.931507 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:22.931441 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="99f22134-a60c-4ed5-9d46-8e0c15815d38" containerName="console" Jul 10 19:04:22.935728 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:22.935709 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lpxzw" Jul 10 19:04:22.938400 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:22.938360 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 10 19:04:22.938400 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:22.938369 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-jxxxx\"" Jul 10 19:04:22.939579 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:22.939560 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 10 19:04:22.944803 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:22.944783 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lpxzw"] Jul 10 19:04:23.041287 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:23.041252 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8fc02889-6e23-4603-bc22-6b7db4163f2d-bound-sa-token\") pod \"cert-manager-55b4d954d5-lpxzw\" (UID: \"8fc02889-6e23-4603-bc22-6b7db4163f2d\") " pod="cert-manager/cert-manager-55b4d954d5-lpxzw" Jul 10 19:04:23.041536 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:23.041371 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45lmf\" (UniqueName: \"kubernetes.io/projected/8fc02889-6e23-4603-bc22-6b7db4163f2d-kube-api-access-45lmf\") pod \"cert-manager-55b4d954d5-lpxzw\" (UID: \"8fc02889-6e23-4603-bc22-6b7db4163f2d\") " pod="cert-manager/cert-manager-55b4d954d5-lpxzw" Jul 10 19:04:23.142184 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:23.142151 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45lmf\" (UniqueName: \"kubernetes.io/projected/8fc02889-6e23-4603-bc22-6b7db4163f2d-kube-api-access-45lmf\") pod \"cert-manager-55b4d954d5-lpxzw\" (UID: \"8fc02889-6e23-4603-bc22-6b7db4163f2d\") " pod="cert-manager/cert-manager-55b4d954d5-lpxzw" Jul 10 19:04:23.142333 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:23.142212 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8fc02889-6e23-4603-bc22-6b7db4163f2d-bound-sa-token\") pod \"cert-manager-55b4d954d5-lpxzw\" (UID: \"8fc02889-6e23-4603-bc22-6b7db4163f2d\") " pod="cert-manager/cert-manager-55b4d954d5-lpxzw" Jul 10 19:04:23.150960 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:23.150932 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8fc02889-6e23-4603-bc22-6b7db4163f2d-bound-sa-token\") pod \"cert-manager-55b4d954d5-lpxzw\" (UID: \"8fc02889-6e23-4603-bc22-6b7db4163f2d\") " pod="cert-manager/cert-manager-55b4d954d5-lpxzw" Jul 10 19:04:23.151060 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:23.151016 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45lmf\" (UniqueName: \"kubernetes.io/projected/8fc02889-6e23-4603-bc22-6b7db4163f2d-kube-api-access-45lmf\") pod \"cert-manager-55b4d954d5-lpxzw\" (UID: \"8fc02889-6e23-4603-bc22-6b7db4163f2d\") " pod="cert-manager/cert-manager-55b4d954d5-lpxzw" Jul 10 19:04:23.254607 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:23.254541 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lpxzw" Jul 10 19:04:23.385918 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:23.385860 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lpxzw"] Jul 10 19:04:23.388790 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:04:23.388762 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8fc02889_6e23_4603_bc22_6b7db4163f2d.slice/crio-f207d8a80700c3a0d170d54c4906032416baa157a15230df4ab275ad07467c44 WatchSource:0}: Error finding container f207d8a80700c3a0d170d54c4906032416baa157a15230df4ab275ad07467c44: Status 404 returned error can't find the container with id f207d8a80700c3a0d170d54c4906032416baa157a15230df4ab275ad07467c44 Jul 10 19:04:24.267956 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:24.267912 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lpxzw" event={"ID":"8fc02889-6e23-4603-bc22-6b7db4163f2d","Type":"ContainerStarted","Data":"f207d8a80700c3a0d170d54c4906032416baa157a15230df4ab275ad07467c44"} Jul 10 19:04:27.284556 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:27.284528 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lpxzw" event={"ID":"8fc02889-6e23-4603-bc22-6b7db4163f2d","Type":"ContainerStarted","Data":"7f51beda99671f3c5c9dd647d6adf89bdef9b34ca8d5e20f8d4f64d6425be380"} Jul 10 19:04:27.301054 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:27.300953 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-lpxzw" podStartSLOduration=1.6900671680000001 podStartE2EDuration="5.300933685s" totalImagesPullingTime="3.610866517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:04:27.300497139 +0000 UTC m=+248.836839572" watchObservedRunningTime="2026-07-10 19:04:27.300933685 +0000 UTC m=+248.837276119" Jul 10 19:04:51.605313 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.605235 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b"] Jul 10 19:04:51.608781 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.608760 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:04:51.611498 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.611471 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 19:04:51.612021 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.612003 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 19:04:51.612754 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.612733 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lhpvl\"" Jul 10 19:04:51.613316 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.613299 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 19:04:51.613424 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.613375 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 19:04:51.633229 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.633207 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b"] Jul 10 19:04:51.668829 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.668806 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c8f9311-d468-4ff4-a392-ca3db133731b-tmp\") pod \"opendatahub-operator-controller-manager-7668cd484b-58f6b\" (UID: \"2c8f9311-d468-4ff4-a392-ca3db133731b\") " pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:04:51.668934 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.668840 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2c8f9311-d468-4ff4-a392-ca3db133731b-manifests\") pod \"opendatahub-operator-controller-manager-7668cd484b-58f6b\" (UID: \"2c8f9311-d468-4ff4-a392-ca3db133731b\") " pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:04:51.668934 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.668859 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmppv\" (UniqueName: \"kubernetes.io/projected/2c8f9311-d468-4ff4-a392-ca3db133731b-kube-api-access-xmppv\") pod \"opendatahub-operator-controller-manager-7668cd484b-58f6b\" (UID: \"2c8f9311-d468-4ff4-a392-ca3db133731b\") " pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:04:51.668934 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.668893 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2c8f9311-d468-4ff4-a392-ca3db133731b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7668cd484b-58f6b\" (UID: \"2c8f9311-d468-4ff4-a392-ca3db133731b\") " pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:04:51.669052 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.668952 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c8f9311-d468-4ff4-a392-ca3db133731b-webhook-cert\") pod \"opendatahub-operator-controller-manager-7668cd484b-58f6b\" (UID: \"2c8f9311-d468-4ff4-a392-ca3db133731b\") " pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:04:51.769594 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.769566 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2c8f9311-d468-4ff4-a392-ca3db133731b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7668cd484b-58f6b\" (UID: \"2c8f9311-d468-4ff4-a392-ca3db133731b\") " pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:04:51.769684 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.769600 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c8f9311-d468-4ff4-a392-ca3db133731b-webhook-cert\") pod \"opendatahub-operator-controller-manager-7668cd484b-58f6b\" (UID: \"2c8f9311-d468-4ff4-a392-ca3db133731b\") " pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:04:51.769684 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.769647 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c8f9311-d468-4ff4-a392-ca3db133731b-tmp\") pod \"opendatahub-operator-controller-manager-7668cd484b-58f6b\" (UID: \"2c8f9311-d468-4ff4-a392-ca3db133731b\") " pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:04:51.769684 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.769672 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2c8f9311-d468-4ff4-a392-ca3db133731b-manifests\") pod \"opendatahub-operator-controller-manager-7668cd484b-58f6b\" (UID: \"2c8f9311-d468-4ff4-a392-ca3db133731b\") " pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:04:51.769840 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.769693 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmppv\" (UniqueName: \"kubernetes.io/projected/2c8f9311-d468-4ff4-a392-ca3db133731b-kube-api-access-xmppv\") pod \"opendatahub-operator-controller-manager-7668cd484b-58f6b\" (UID: \"2c8f9311-d468-4ff4-a392-ca3db133731b\") " pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:04:51.770077 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.770058 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2c8f9311-d468-4ff4-a392-ca3db133731b-manifests\") pod \"opendatahub-operator-controller-manager-7668cd484b-58f6b\" (UID: \"2c8f9311-d468-4ff4-a392-ca3db133731b\") " pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:04:51.771863 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.771840 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c8f9311-d468-4ff4-a392-ca3db133731b-tmp\") pod \"opendatahub-operator-controller-manager-7668cd484b-58f6b\" (UID: \"2c8f9311-d468-4ff4-a392-ca3db133731b\") " pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:04:51.771956 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.771907 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2c8f9311-d468-4ff4-a392-ca3db133731b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7668cd484b-58f6b\" (UID: \"2c8f9311-d468-4ff4-a392-ca3db133731b\") " pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:04:51.772024 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.772008 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c8f9311-d468-4ff4-a392-ca3db133731b-webhook-cert\") pod \"opendatahub-operator-controller-manager-7668cd484b-58f6b\" (UID: \"2c8f9311-d468-4ff4-a392-ca3db133731b\") " pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:04:51.777960 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.777941 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmppv\" (UniqueName: \"kubernetes.io/projected/2c8f9311-d468-4ff4-a392-ca3db133731b-kube-api-access-xmppv\") pod \"opendatahub-operator-controller-manager-7668cd484b-58f6b\" (UID: \"2c8f9311-d468-4ff4-a392-ca3db133731b\") " pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:04:51.919519 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:51.919461 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:04:52.066278 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:52.066249 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b"] Jul 10 19:04:52.070331 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:04:52.070293 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c8f9311_d468_4ff4_a392_ca3db133731b.slice/crio-eb8b7ff793aebec4f00e880178394bc7f47375951f8704cdb59f96bf76200662 WatchSource:0}: Error finding container eb8b7ff793aebec4f00e880178394bc7f47375951f8704cdb59f96bf76200662: Status 404 returned error can't find the container with id eb8b7ff793aebec4f00e880178394bc7f47375951f8704cdb59f96bf76200662 Jul 10 19:04:52.389170 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:52.389136 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" event={"ID":"2c8f9311-d468-4ff4-a392-ca3db133731b","Type":"ContainerStarted","Data":"eb8b7ff793aebec4f00e880178394bc7f47375951f8704cdb59f96bf76200662"} Jul 10 19:04:55.405892 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:55.405849 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" event={"ID":"2c8f9311-d468-4ff4-a392-ca3db133731b","Type":"ContainerStarted","Data":"afb5a65035b394d852400ebde1f2c54ea284307a3984af6273fd6b8fbef4e4fe"} Jul 10 19:04:56.411862 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:56.411828 2571 generic.go:356] "Generic (PLEG): container finished" podID="2c8f9311-d468-4ff4-a392-ca3db133731b" containerID="afb5a65035b394d852400ebde1f2c54ea284307a3984af6273fd6b8fbef4e4fe" exitCode=0 Jul 10 19:04:56.412275 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:56.411871 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" event={"ID":"2c8f9311-d468-4ff4-a392-ca3db133731b","Type":"ContainerDied","Data":"afb5a65035b394d852400ebde1f2c54ea284307a3984af6273fd6b8fbef4e4fe"} Jul 10 19:04:57.418019 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:57.417987 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" event={"ID":"2c8f9311-d468-4ff4-a392-ca3db133731b","Type":"ContainerStarted","Data":"1cb824ee5ca7b920df32de03203135353919ca1c376cc5f83d0a08cdd57a145b"} Jul 10 19:04:57.418489 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:57.418134 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:04:57.441330 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:04:57.441276 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" podStartSLOduration=1.934172909 podStartE2EDuration="6.44126006s" totalImagesPullingTime="3.488315365s" totalInitContainerRuntime="1.018771786s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:04:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:04:57.4404296 +0000 UTC m=+278.976772032" watchObservedRunningTime="2026-07-10 19:04:57.44126006 +0000 UTC m=+278.977602486" Jul 10 19:05:08.424208 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:08.424169 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7668cd484b-58f6b" Jul 10 19:05:18.899376 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:18.899347 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:05:18.906504 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:18.906485 2571 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 19:05:18.906620 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:18.906518 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:05:19.572909 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.572863 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz"] Jul 10 19:05:19.574975 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.574957 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" Jul 10 19:05:19.577796 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.577771 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 10 19:05:19.578959 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.578939 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2tct4\"" Jul 10 19:05:19.579078 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.578973 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 10 19:05:19.579078 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.578953 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 10 19:05:19.579078 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.578954 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 10 19:05:19.579078 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.578956 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 10 19:05:19.588606 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.588581 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz"] Jul 10 19:05:19.705236 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.705199 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/44a59330-6b55-4e66-b169-71101482d022-metrics-cert\") pod \"lws-controller-manager-6d9b94cf9d-cv2fz\" (UID: \"44a59330-6b55-4e66-b169-71101482d022\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" Jul 10 19:05:19.705411 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.705259 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z9m8\" (UniqueName: \"kubernetes.io/projected/44a59330-6b55-4e66-b169-71101482d022-kube-api-access-7z9m8\") pod \"lws-controller-manager-6d9b94cf9d-cv2fz\" (UID: \"44a59330-6b55-4e66-b169-71101482d022\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" Jul 10 19:05:19.705411 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.705327 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/44a59330-6b55-4e66-b169-71101482d022-manager-config\") pod \"lws-controller-manager-6d9b94cf9d-cv2fz\" (UID: \"44a59330-6b55-4e66-b169-71101482d022\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" Jul 10 19:05:19.705411 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.705385 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44a59330-6b55-4e66-b169-71101482d022-cert\") pod \"lws-controller-manager-6d9b94cf9d-cv2fz\" (UID: \"44a59330-6b55-4e66-b169-71101482d022\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" Jul 10 19:05:19.805860 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.805822 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/44a59330-6b55-4e66-b169-71101482d022-manager-config\") pod \"lws-controller-manager-6d9b94cf9d-cv2fz\" (UID: \"44a59330-6b55-4e66-b169-71101482d022\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" Jul 10 19:05:19.806035 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.805893 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44a59330-6b55-4e66-b169-71101482d022-cert\") pod \"lws-controller-manager-6d9b94cf9d-cv2fz\" (UID: \"44a59330-6b55-4e66-b169-71101482d022\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" Jul 10 19:05:19.806035 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.805950 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/44a59330-6b55-4e66-b169-71101482d022-metrics-cert\") pod \"lws-controller-manager-6d9b94cf9d-cv2fz\" (UID: \"44a59330-6b55-4e66-b169-71101482d022\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" Jul 10 19:05:19.806035 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.805972 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z9m8\" (UniqueName: \"kubernetes.io/projected/44a59330-6b55-4e66-b169-71101482d022-kube-api-access-7z9m8\") pod \"lws-controller-manager-6d9b94cf9d-cv2fz\" (UID: \"44a59330-6b55-4e66-b169-71101482d022\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" Jul 10 19:05:19.806975 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.806946 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/44a59330-6b55-4e66-b169-71101482d022-manager-config\") pod \"lws-controller-manager-6d9b94cf9d-cv2fz\" (UID: \"44a59330-6b55-4e66-b169-71101482d022\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" Jul 10 19:05:19.809280 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.809251 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/44a59330-6b55-4e66-b169-71101482d022-metrics-cert\") pod \"lws-controller-manager-6d9b94cf9d-cv2fz\" (UID: \"44a59330-6b55-4e66-b169-71101482d022\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" Jul 10 19:05:19.809397 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.809322 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44a59330-6b55-4e66-b169-71101482d022-cert\") pod \"lws-controller-manager-6d9b94cf9d-cv2fz\" (UID: \"44a59330-6b55-4e66-b169-71101482d022\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" Jul 10 19:05:19.836619 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.836557 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z9m8\" (UniqueName: \"kubernetes.io/projected/44a59330-6b55-4e66-b169-71101482d022-kube-api-access-7z9m8\") pod \"lws-controller-manager-6d9b94cf9d-cv2fz\" (UID: \"44a59330-6b55-4e66-b169-71101482d022\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" Jul 10 19:05:19.884939 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:19.884905 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" Jul 10 19:05:20.020156 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:20.020131 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz"] Jul 10 19:05:20.022457 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:05:20.022431 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44a59330_6b55_4e66_b169_71101482d022.slice/crio-1be5002395ead480787c4af11b358ddd431b8851aa6b0bf54200d8ecd8c58a9a WatchSource:0}: Error finding container 1be5002395ead480787c4af11b358ddd431b8851aa6b0bf54200d8ecd8c58a9a: Status 404 returned error can't find the container with id 1be5002395ead480787c4af11b358ddd431b8851aa6b0bf54200d8ecd8c58a9a Jul 10 19:05:20.024390 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:20.024373 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 19:05:20.519434 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:20.519396 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" event={"ID":"44a59330-6b55-4e66-b169-71101482d022","Type":"ContainerStarted","Data":"1be5002395ead480787c4af11b358ddd431b8851aa6b0bf54200d8ecd8c58a9a"} Jul 10 19:05:22.499068 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.499036 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-pmzsc"] Jul 10 19:05:22.501669 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.501647 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-b92cg"] Jul 10 19:05:22.501771 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.501751 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pmzsc" Jul 10 19:05:22.505046 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.505022 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 10 19:05:22.505201 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.505056 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-l99bb\"" Jul 10 19:05:22.505201 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.505122 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b92cg" Jul 10 19:05:22.508577 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.508556 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-hgmdr\"" Jul 10 19:05:22.508820 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.508805 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 10 19:05:22.518336 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.518312 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-pmzsc"] Jul 10 19:05:22.520508 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.520485 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-b92cg"] Jul 10 19:05:22.636295 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.636255 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxzhx\" (UniqueName: \"kubernetes.io/projected/5112fd0d-a35d-4710-9fbc-9ba814d56e22-kube-api-access-gxzhx\") pod \"llmisvc-controller-manager-777c6bd7d5-b92cg\" (UID: \"5112fd0d-a35d-4710-9fbc-9ba814d56e22\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b92cg" Jul 10 19:05:22.636442 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.636320 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pphgj\" (UniqueName: \"kubernetes.io/projected/4bbbb813-070d-4dfa-ada4-0a72be142c3b-kube-api-access-pphgj\") pod \"kserve-controller-manager-d6f4b65b8-pmzsc\" (UID: \"4bbbb813-070d-4dfa-ada4-0a72be142c3b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pmzsc" Jul 10 19:05:22.636442 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.636373 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5112fd0d-a35d-4710-9fbc-9ba814d56e22-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-b92cg\" (UID: \"5112fd0d-a35d-4710-9fbc-9ba814d56e22\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b92cg" Jul 10 19:05:22.636536 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.636462 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4bbbb813-070d-4dfa-ada4-0a72be142c3b-cert\") pod \"kserve-controller-manager-d6f4b65b8-pmzsc\" (UID: \"4bbbb813-070d-4dfa-ada4-0a72be142c3b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pmzsc" Jul 10 19:05:22.737617 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.737589 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5112fd0d-a35d-4710-9fbc-9ba814d56e22-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-b92cg\" (UID: \"5112fd0d-a35d-4710-9fbc-9ba814d56e22\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b92cg" Jul 10 19:05:22.737759 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.737656 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4bbbb813-070d-4dfa-ada4-0a72be142c3b-cert\") pod \"kserve-controller-manager-d6f4b65b8-pmzsc\" (UID: \"4bbbb813-070d-4dfa-ada4-0a72be142c3b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pmzsc" Jul 10 19:05:22.737759 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:05:22.737752 2571 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 10 19:05:22.737837 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.737752 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxzhx\" (UniqueName: \"kubernetes.io/projected/5112fd0d-a35d-4710-9fbc-9ba814d56e22-kube-api-access-gxzhx\") pod \"llmisvc-controller-manager-777c6bd7d5-b92cg\" (UID: \"5112fd0d-a35d-4710-9fbc-9ba814d56e22\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b92cg" Jul 10 19:05:22.737837 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:05:22.737809 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bbbb813-070d-4dfa-ada4-0a72be142c3b-cert podName:4bbbb813-070d-4dfa-ada4-0a72be142c3b nodeName:}" failed. No retries permitted until 2026-07-10 19:05:23.23778975 +0000 UTC m=+304.774132160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4bbbb813-070d-4dfa-ada4-0a72be142c3b-cert") pod "kserve-controller-manager-d6f4b65b8-pmzsc" (UID: "4bbbb813-070d-4dfa-ada4-0a72be142c3b") : secret "kserve-webhook-server-cert" not found Jul 10 19:05:22.737923 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.737862 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pphgj\" (UniqueName: \"kubernetes.io/projected/4bbbb813-070d-4dfa-ada4-0a72be142c3b-kube-api-access-pphgj\") pod \"kserve-controller-manager-d6f4b65b8-pmzsc\" (UID: \"4bbbb813-070d-4dfa-ada4-0a72be142c3b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pmzsc" Jul 10 19:05:22.739950 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.739930 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5112fd0d-a35d-4710-9fbc-9ba814d56e22-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-b92cg\" (UID: \"5112fd0d-a35d-4710-9fbc-9ba814d56e22\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b92cg" Jul 10 19:05:22.769911 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.769852 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxzhx\" (UniqueName: \"kubernetes.io/projected/5112fd0d-a35d-4710-9fbc-9ba814d56e22-kube-api-access-gxzhx\") pod \"llmisvc-controller-manager-777c6bd7d5-b92cg\" (UID: \"5112fd0d-a35d-4710-9fbc-9ba814d56e22\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b92cg" Jul 10 19:05:22.771140 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.771117 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pphgj\" (UniqueName: \"kubernetes.io/projected/4bbbb813-070d-4dfa-ada4-0a72be142c3b-kube-api-access-pphgj\") pod \"kserve-controller-manager-d6f4b65b8-pmzsc\" (UID: \"4bbbb813-070d-4dfa-ada4-0a72be142c3b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pmzsc" Jul 10 19:05:22.823384 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.823341 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b92cg" Jul 10 19:05:22.974689 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:22.974662 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-b92cg"] Jul 10 19:05:22.977991 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:05:22.977964 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5112fd0d_a35d_4710_9fbc_9ba814d56e22.slice/crio-130bdd503d9fa8f1502f768c6fc21e9c9a8255f4a5dbe82732175fd49299964a WatchSource:0}: Error finding container 130bdd503d9fa8f1502f768c6fc21e9c9a8255f4a5dbe82732175fd49299964a: Status 404 returned error can't find the container with id 130bdd503d9fa8f1502f768c6fc21e9c9a8255f4a5dbe82732175fd49299964a Jul 10 19:05:23.241868 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:23.241840 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4bbbb813-070d-4dfa-ada4-0a72be142c3b-cert\") pod \"kserve-controller-manager-d6f4b65b8-pmzsc\" (UID: \"4bbbb813-070d-4dfa-ada4-0a72be142c3b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pmzsc" Jul 10 19:05:23.244271 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:23.244244 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4bbbb813-070d-4dfa-ada4-0a72be142c3b-cert\") pod \"kserve-controller-manager-d6f4b65b8-pmzsc\" (UID: \"4bbbb813-070d-4dfa-ada4-0a72be142c3b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pmzsc" Jul 10 19:05:23.416903 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:23.416875 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pmzsc" Jul 10 19:05:23.534646 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:23.534613 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" event={"ID":"44a59330-6b55-4e66-b169-71101482d022","Type":"ContainerStarted","Data":"a6de1f081f3f8b7fb8c5a17b922df22bfc9d1a85bac2be858b9caca681057e42"} Jul 10 19:05:23.535060 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:23.534764 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" Jul 10 19:05:23.535807 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:23.535786 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b92cg" event={"ID":"5112fd0d-a35d-4710-9fbc-9ba814d56e22","Type":"ContainerStarted","Data":"130bdd503d9fa8f1502f768c6fc21e9c9a8255f4a5dbe82732175fd49299964a"} Jul 10 19:05:23.563343 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:23.563325 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-pmzsc"] Jul 10 19:05:23.565598 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:05:23.565575 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bbbb813_070d_4dfa_ada4_0a72be142c3b.slice/crio-acfc1ba8bd5d95c8ee582b34bbeabcb401c0e1792d9b9023f6124749b8f661d2 WatchSource:0}: Error finding container acfc1ba8bd5d95c8ee582b34bbeabcb401c0e1792d9b9023f6124749b8f661d2: Status 404 returned error can't find the container with id acfc1ba8bd5d95c8ee582b34bbeabcb401c0e1792d9b9023f6124749b8f661d2 Jul 10 19:05:23.607052 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:23.607003 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" podStartSLOduration=2.099890528 podStartE2EDuration="4.606988869s" totalImagesPullingTime="2.507098341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:05:23.605218671 +0000 UTC m=+305.141561102" watchObservedRunningTime="2026-07-10 19:05:23.606988869 +0000 UTC m=+305.143331370" Jul 10 19:05:24.543534 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:24.543495 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pmzsc" event={"ID":"4bbbb813-070d-4dfa-ada4-0a72be142c3b","Type":"ContainerStarted","Data":"acfc1ba8bd5d95c8ee582b34bbeabcb401c0e1792d9b9023f6124749b8f661d2"} Jul 10 19:05:26.554320 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:26.554295 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b92cg" event={"ID":"5112fd0d-a35d-4710-9fbc-9ba814d56e22","Type":"ContainerStarted","Data":"c88a6f45afd4de9d3d2d3d0cdd83c127faf05bd6845185554483a847dc8a980b"} Jul 10 19:05:26.555910 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:26.555887 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b92cg" Jul 10 19:05:26.587050 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:26.586999 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b92cg" podStartSLOduration=1.759762335 podStartE2EDuration="4.586979262s" totalImagesPullingTime="2.827216927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:05:26.585592511 +0000 UTC m=+308.121934944" watchObservedRunningTime="2026-07-10 19:05:26.586979262 +0000 UTC m=+308.123321696" Jul 10 19:05:27.562935 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:27.562894 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pmzsc" event={"ID":"4bbbb813-070d-4dfa-ada4-0a72be142c3b","Type":"ContainerStarted","Data":"a6d4d5c10914fb9e8e566810813573d22cf85d9b4196463fbe5175b8e621d818"} Jul 10 19:05:27.563427 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:27.563143 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pmzsc" Jul 10 19:05:27.597646 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:27.597603 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pmzsc" podStartSLOduration=2.613041976 podStartE2EDuration="5.597587164s" totalImagesPullingTime="2.984545188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:05:27.596049719 +0000 UTC m=+309.132392151" watchObservedRunningTime="2026-07-10 19:05:27.597587164 +0000 UTC m=+309.133929596" Jul 10 19:05:34.545867 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:34.545838 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6d9b94cf9d-cv2fz" Jul 10 19:05:39.283450 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.283419 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt"] Jul 10 19:05:39.290152 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.290131 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.292914 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.292892 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 19:05:39.293039 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.293018 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-csnjt\"" Jul 10 19:05:39.312554 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.312527 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt"] Jul 10 19:05:39.384307 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.384282 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.384425 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.384313 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.384425 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.384357 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.384425 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.384385 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.384425 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.384406 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.384558 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.384428 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.384558 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.384503 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.384558 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.384533 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d258153-2e48-4df4-90dc-e1bba1a245a5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.384558 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.384555 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6xlp\" (UniqueName: \"kubernetes.io/projected/9d258153-2e48-4df4-90dc-e1bba1a245a5-kube-api-access-j6xlp\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.485869 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.485836 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.485869 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.485872 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.486044 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.485906 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.486044 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.485926 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.486044 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.485950 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.486044 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.485979 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.486044 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.486016 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.486044 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.486039 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d258153-2e48-4df4-90dc-e1bba1a245a5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.486336 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.486138 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6xlp\" (UniqueName: \"kubernetes.io/projected/9d258153-2e48-4df4-90dc-e1bba1a245a5-kube-api-access-j6xlp\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.486336 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.486308 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.486417 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.486359 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.486500 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.486479 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.486558 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.486536 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.486851 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.486829 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d258153-2e48-4df4-90dc-e1bba1a245a5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.488395 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.488376 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.488497 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.488479 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.494566 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.494537 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6xlp\" (UniqueName: \"kubernetes.io/projected/9d258153-2e48-4df4-90dc-e1bba1a245a5-kube-api-access-j6xlp\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.495262 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.495237 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-548557588-x5gvt\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.601455 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.601390 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:39.740478 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:39.740348 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt"] Jul 10 19:05:39.743285 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:05:39.743255 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d258153_2e48_4df4_90dc_e1bba1a245a5.slice/crio-8bfad490693da94ed2e719517ac5712c7b3d6e89ca584764fc66665b7aff78cd WatchSource:0}: Error finding container 8bfad490693da94ed2e719517ac5712c7b3d6e89ca584764fc66665b7aff78cd: Status 404 returned error can't find the container with id 8bfad490693da94ed2e719517ac5712c7b3d6e89ca584764fc66665b7aff78cd Jul 10 19:05:40.626872 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:40.626831 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" event={"ID":"9d258153-2e48-4df4-90dc-e1bba1a245a5","Type":"ContainerStarted","Data":"8bfad490693da94ed2e719517ac5712c7b3d6e89ca584764fc66665b7aff78cd"} Jul 10 19:05:42.301480 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:42.301442 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 19:05:42.301741 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:42.301513 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 19:05:42.301741 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:42.301547 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 19:05:42.638305 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:42.638215 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" event={"ID":"9d258153-2e48-4df4-90dc-e1bba1a245a5","Type":"ContainerStarted","Data":"8be17bee4645aa7f053e3ae61c8976e92f2c6e91c443bbfb648197197d64759a"} Jul 10 19:05:42.662451 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:42.662398 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" podStartSLOduration=1.106387682 podStartE2EDuration="3.662380398s" totalImagesPullingTime="2.555992716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:05:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:05:42.660835907 +0000 UTC m=+324.197178363" watchObservedRunningTime="2026-07-10 19:05:42.662380398 +0000 UTC m=+324.198722830" Jul 10 19:05:43.602173 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:43.602135 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:43.603454 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:43.603428 2571 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-548557588-x5gvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 10 19:05:43.603512 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:43.603489 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" podUID="9d258153-2e48-4df4-90dc-e1bba1a245a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 10 19:05:44.356310 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:44.356270 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt"] Jul 10 19:05:44.602016 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:44.601980 2571 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-548557588-x5gvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 10 19:05:44.602192 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:44.602044 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" podUID="9d258153-2e48-4df4-90dc-e1bba1a245a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 10 19:05:45.602121 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:45.602069 2571 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-548557588-x5gvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 10 19:05:45.602296 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:45.602144 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" podUID="9d258153-2e48-4df4-90dc-e1bba1a245a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 10 19:05:45.652819 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:45.652783 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" podUID="9d258153-2e48-4df4-90dc-e1bba1a245a5" containerName="istio-proxy" containerID="cri-o://8be17bee4645aa7f053e3ae61c8976e92f2c6e91c443bbfb648197197d64759a" gracePeriod=30 Jul 10 19:05:50.908362 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.908337 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:50.993669 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.993601 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-token\") pod \"9d258153-2e48-4df4-90dc-e1bba1a245a5\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " Jul 10 19:05:50.993669 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.993636 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d258153-2e48-4df4-90dc-e1bba1a245a5-istiod-ca-cert\") pod \"9d258153-2e48-4df4-90dc-e1bba1a245a5\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " Jul 10 19:05:50.993669 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.993664 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-envoy\") pod \"9d258153-2e48-4df4-90dc-e1bba1a245a5\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " Jul 10 19:05:50.994242 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.993695 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-data\") pod \"9d258153-2e48-4df4-90dc-e1bba1a245a5\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " Jul 10 19:05:50.994242 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.993717 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-workload-socket\") pod \"9d258153-2e48-4df4-90dc-e1bba1a245a5\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " Jul 10 19:05:50.994242 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.993735 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-podinfo\") pod \"9d258153-2e48-4df4-90dc-e1bba1a245a5\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " Jul 10 19:05:50.994242 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.993787 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6xlp\" (UniqueName: \"kubernetes.io/projected/9d258153-2e48-4df4-90dc-e1bba1a245a5-kube-api-access-j6xlp\") pod \"9d258153-2e48-4df4-90dc-e1bba1a245a5\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " Jul 10 19:05:50.994242 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.993844 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-workload-certs\") pod \"9d258153-2e48-4df4-90dc-e1bba1a245a5\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " Jul 10 19:05:50.994242 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.993893 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-credential-socket\") pod \"9d258153-2e48-4df4-90dc-e1bba1a245a5\" (UID: \"9d258153-2e48-4df4-90dc-e1bba1a245a5\") " Jul 10 19:05:50.994242 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.994044 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-data" (OuterVolumeSpecName: "istio-data") pod "9d258153-2e48-4df4-90dc-e1bba1a245a5" (UID: "9d258153-2e48-4df4-90dc-e1bba1a245a5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:05:50.994242 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.993992 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d258153-2e48-4df4-90dc-e1bba1a245a5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9d258153-2e48-4df4-90dc-e1bba1a245a5" (UID: "9d258153-2e48-4df4-90dc-e1bba1a245a5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:05:50.994242 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.994087 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9d258153-2e48-4df4-90dc-e1bba1a245a5" (UID: "9d258153-2e48-4df4-90dc-e1bba1a245a5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:05:50.994242 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.994207 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d258153-2e48-4df4-90dc-e1bba1a245a5-istiod-ca-cert\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:05:50.994242 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.994226 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-data\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:05:50.994242 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.994238 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-workload-socket\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:05:50.994868 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.994276 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9d258153-2e48-4df4-90dc-e1bba1a245a5" (UID: "9d258153-2e48-4df4-90dc-e1bba1a245a5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:05:50.994868 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.994293 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9d258153-2e48-4df4-90dc-e1bba1a245a5" (UID: "9d258153-2e48-4df4-90dc-e1bba1a245a5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:05:50.995949 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.995923 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-token" (OuterVolumeSpecName: "istio-token") pod "9d258153-2e48-4df4-90dc-e1bba1a245a5" (UID: "9d258153-2e48-4df4-90dc-e1bba1a245a5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:05:50.996042 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.995961 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9d258153-2e48-4df4-90dc-e1bba1a245a5" (UID: "9d258153-2e48-4df4-90dc-e1bba1a245a5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 19:05:50.996042 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.995993 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d258153-2e48-4df4-90dc-e1bba1a245a5-kube-api-access-j6xlp" (OuterVolumeSpecName: "kube-api-access-j6xlp") pod "9d258153-2e48-4df4-90dc-e1bba1a245a5" (UID: "9d258153-2e48-4df4-90dc-e1bba1a245a5"). InnerVolumeSpecName "kube-api-access-j6xlp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:05:50.996156 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:50.996060 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9d258153-2e48-4df4-90dc-e1bba1a245a5" (UID: "9d258153-2e48-4df4-90dc-e1bba1a245a5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:05:51.095469 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:51.095433 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-credential-socket\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:05:51.095469 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:51.095465 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-token\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:05:51.095469 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:51.095475 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-envoy\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:05:51.095715 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:51.095484 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d258153-2e48-4df4-90dc-e1bba1a245a5-istio-podinfo\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:05:51.095715 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:51.095492 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6xlp\" (UniqueName: \"kubernetes.io/projected/9d258153-2e48-4df4-90dc-e1bba1a245a5-kube-api-access-j6xlp\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:05:51.095715 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:51.095501 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d258153-2e48-4df4-90dc-e1bba1a245a5-workload-certs\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:05:51.680623 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:51.680590 2571 generic.go:356] "Generic (PLEG): container finished" podID="9d258153-2e48-4df4-90dc-e1bba1a245a5" containerID="8be17bee4645aa7f053e3ae61c8976e92f2c6e91c443bbfb648197197d64759a" exitCode=0 Jul 10 19:05:51.680623 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:51.680628 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" event={"ID":"9d258153-2e48-4df4-90dc-e1bba1a245a5","Type":"ContainerDied","Data":"8be17bee4645aa7f053e3ae61c8976e92f2c6e91c443bbfb648197197d64759a"} Jul 10 19:05:51.680828 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:51.680638 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" Jul 10 19:05:51.680828 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:51.680670 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt" event={"ID":"9d258153-2e48-4df4-90dc-e1bba1a245a5","Type":"ContainerDied","Data":"8bfad490693da94ed2e719517ac5712c7b3d6e89ca584764fc66665b7aff78cd"} Jul 10 19:05:51.680828 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:51.680691 2571 scope.go:117] "RemoveContainer" containerID="8be17bee4645aa7f053e3ae61c8976e92f2c6e91c443bbfb648197197d64759a" Jul 10 19:05:51.689535 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:51.689515 2571 scope.go:117] "RemoveContainer" containerID="8be17bee4645aa7f053e3ae61c8976e92f2c6e91c443bbfb648197197d64759a" Jul 10 19:05:51.689831 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:05:51.689804 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8be17bee4645aa7f053e3ae61c8976e92f2c6e91c443bbfb648197197d64759a\": container with ID starting with 8be17bee4645aa7f053e3ae61c8976e92f2c6e91c443bbfb648197197d64759a not found: ID does not exist" containerID="8be17bee4645aa7f053e3ae61c8976e92f2c6e91c443bbfb648197197d64759a" Jul 10 19:05:51.689885 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:51.689839 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8be17bee4645aa7f053e3ae61c8976e92f2c6e91c443bbfb648197197d64759a"} err="failed to get container status \"8be17bee4645aa7f053e3ae61c8976e92f2c6e91c443bbfb648197197d64759a\": rpc error: code = NotFound desc = could not find container \"8be17bee4645aa7f053e3ae61c8976e92f2c6e91c443bbfb648197197d64759a\": container with ID starting with 8be17bee4645aa7f053e3ae61c8976e92f2c6e91c443bbfb648197197d64759a not found: ID does not exist" Jul 10 19:05:51.700701 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:51.700680 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt"] Jul 10 19:05:51.705033 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:51.705014 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-x5gvt"] Jul 10 19:05:53.017708 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:53.017674 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d258153-2e48-4df4-90dc-e1bba1a245a5" path="/var/lib/kubelet/pods/9d258153-2e48-4df4-90dc-e1bba1a245a5/volumes" Jul 10 19:05:57.565309 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:57.565279 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b92cg" Jul 10 19:05:58.572157 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:05:58.572121 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pmzsc" Jul 10 19:06:12.513260 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:12.513174 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pcdzp"] Jul 10 19:06:12.513960 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:12.513562 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d258153-2e48-4df4-90dc-e1bba1a245a5" containerName="istio-proxy" Jul 10 19:06:12.513960 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:12.513576 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d258153-2e48-4df4-90dc-e1bba1a245a5" containerName="istio-proxy" Jul 10 19:06:12.513960 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:12.513647 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d258153-2e48-4df4-90dc-e1bba1a245a5" containerName="istio-proxy" Jul 10 19:06:12.516616 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:12.516597 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pcdzp" Jul 10 19:06:12.521654 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:12.521630 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 19:06:12.522897 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:12.522872 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 19:06:12.523032 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:12.522878 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-k8sx2\"" Jul 10 19:06:12.540453 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:12.540426 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pcdzp"] Jul 10 19:06:12.577376 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:12.577345 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6phc\" (UniqueName: \"kubernetes.io/projected/87fe3dda-1ba8-46e4-a886-056b366a0fa1-kube-api-access-h6phc\") pod \"kuadrant-operator-catalog-pcdzp\" (UID: \"87fe3dda-1ba8-46e4-a886-056b366a0fa1\") " pod="kuadrant-system/kuadrant-operator-catalog-pcdzp" Jul 10 19:06:12.678109 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:12.678058 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6phc\" (UniqueName: \"kubernetes.io/projected/87fe3dda-1ba8-46e4-a886-056b366a0fa1-kube-api-access-h6phc\") pod \"kuadrant-operator-catalog-pcdzp\" (UID: \"87fe3dda-1ba8-46e4-a886-056b366a0fa1\") " pod="kuadrant-system/kuadrant-operator-catalog-pcdzp" Jul 10 19:06:12.687186 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:12.687156 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6phc\" (UniqueName: \"kubernetes.io/projected/87fe3dda-1ba8-46e4-a886-056b366a0fa1-kube-api-access-h6phc\") pod \"kuadrant-operator-catalog-pcdzp\" (UID: \"87fe3dda-1ba8-46e4-a886-056b366a0fa1\") " pod="kuadrant-system/kuadrant-operator-catalog-pcdzp" Jul 10 19:06:12.827069 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:12.827043 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pcdzp" Jul 10 19:06:12.868982 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:12.868951 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pcdzp"] Jul 10 19:06:12.959614 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:12.959588 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pcdzp"] Jul 10 19:06:12.961748 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:06:12.961706 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87fe3dda_1ba8_46e4_a886_056b366a0fa1.slice/crio-ab4067b5715ce3514ba7e39d1fba1d9e7235b1c9a497265bc87526a67c21627b WatchSource:0}: Error finding container ab4067b5715ce3514ba7e39d1fba1d9e7235b1c9a497265bc87526a67c21627b: Status 404 returned error can't find the container with id ab4067b5715ce3514ba7e39d1fba1d9e7235b1c9a497265bc87526a67c21627b Jul 10 19:06:13.077801 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:13.077739 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-btldc"] Jul 10 19:06:13.082270 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:13.082247 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-btldc" Jul 10 19:06:13.101968 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:13.101942 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-btldc"] Jul 10 19:06:13.183504 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:13.183474 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99c6h\" (UniqueName: \"kubernetes.io/projected/0aaea817-77c7-4656-a317-da611d831777-kube-api-access-99c6h\") pod \"kuadrant-operator-catalog-btldc\" (UID: \"0aaea817-77c7-4656-a317-da611d831777\") " pod="kuadrant-system/kuadrant-operator-catalog-btldc" Jul 10 19:06:13.284856 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:13.284811 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99c6h\" (UniqueName: \"kubernetes.io/projected/0aaea817-77c7-4656-a317-da611d831777-kube-api-access-99c6h\") pod \"kuadrant-operator-catalog-btldc\" (UID: \"0aaea817-77c7-4656-a317-da611d831777\") " pod="kuadrant-system/kuadrant-operator-catalog-btldc" Jul 10 19:06:13.294189 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:13.294170 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99c6h\" (UniqueName: \"kubernetes.io/projected/0aaea817-77c7-4656-a317-da611d831777-kube-api-access-99c6h\") pod \"kuadrant-operator-catalog-btldc\" (UID: \"0aaea817-77c7-4656-a317-da611d831777\") " pod="kuadrant-system/kuadrant-operator-catalog-btldc" Jul 10 19:06:13.391457 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:13.391382 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-btldc" Jul 10 19:06:13.521734 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:13.521705 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-btldc"] Jul 10 19:06:13.524694 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:06:13.524672 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aaea817_77c7_4656_a317_da611d831777.slice/crio-5729f06ee5dac50a2f3dd2bfbdafd806b63617e101e3f885e734b910a8122f28 WatchSource:0}: Error finding container 5729f06ee5dac50a2f3dd2bfbdafd806b63617e101e3f885e734b910a8122f28: Status 404 returned error can't find the container with id 5729f06ee5dac50a2f3dd2bfbdafd806b63617e101e3f885e734b910a8122f28 Jul 10 19:06:13.782120 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:13.782046 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-btldc" event={"ID":"0aaea817-77c7-4656-a317-da611d831777","Type":"ContainerStarted","Data":"5729f06ee5dac50a2f3dd2bfbdafd806b63617e101e3f885e734b910a8122f28"} Jul 10 19:06:13.783286 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:13.783263 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pcdzp" event={"ID":"87fe3dda-1ba8-46e4-a886-056b366a0fa1","Type":"ContainerStarted","Data":"ab4067b5715ce3514ba7e39d1fba1d9e7235b1c9a497265bc87526a67c21627b"} Jul 10 19:06:15.793891 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:15.793849 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pcdzp" event={"ID":"87fe3dda-1ba8-46e4-a886-056b366a0fa1","Type":"ContainerStarted","Data":"c457a7cbdef514b665a2e827f416791202d92da5206f5904fbea9ac84840e1ea"} Jul 10 19:06:15.794347 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:15.793855 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-pcdzp" podUID="87fe3dda-1ba8-46e4-a886-056b366a0fa1" containerName="registry-server" containerID="cri-o://c457a7cbdef514b665a2e827f416791202d92da5206f5904fbea9ac84840e1ea" gracePeriod=2 Jul 10 19:06:15.795357 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:15.795335 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-btldc" event={"ID":"0aaea817-77c7-4656-a317-da611d831777","Type":"ContainerStarted","Data":"69e3f9289b53cfa1bda30f956236c9ee2ec10e45aea5eaba267bdd00118d82e5"} Jul 10 19:06:15.812615 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:15.812576 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-pcdzp" podStartSLOduration=1.554740681 podStartE2EDuration="3.812563685s" totalImagesPullingTime="2.257823004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:06:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:06:15.810471816 +0000 UTC m=+357.346814248" watchObservedRunningTime="2026-07-10 19:06:15.812563685 +0000 UTC m=+357.348906118" Jul 10 19:06:15.827911 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:15.827869 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-btldc" podStartSLOduration=1.132433166 podStartE2EDuration="2.827856326s" totalImagesPullingTime="1.69542316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:06:15.826157292 +0000 UTC m=+357.362499724" watchObservedRunningTime="2026-07-10 19:06:15.827856326 +0000 UTC m=+357.364198758" Jul 10 19:06:16.044149 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:16.044081 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pcdzp" Jul 10 19:06:16.111666 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:16.111641 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6phc\" (UniqueName: \"kubernetes.io/projected/87fe3dda-1ba8-46e4-a886-056b366a0fa1-kube-api-access-h6phc\") pod \"87fe3dda-1ba8-46e4-a886-056b366a0fa1\" (UID: \"87fe3dda-1ba8-46e4-a886-056b366a0fa1\") " Jul 10 19:06:16.113686 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:16.113663 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87fe3dda-1ba8-46e4-a886-056b366a0fa1-kube-api-access-h6phc" (OuterVolumeSpecName: "kube-api-access-h6phc") pod "87fe3dda-1ba8-46e4-a886-056b366a0fa1" (UID: "87fe3dda-1ba8-46e4-a886-056b366a0fa1"). InnerVolumeSpecName "kube-api-access-h6phc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:06:16.212285 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:16.212260 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6phc\" (UniqueName: \"kubernetes.io/projected/87fe3dda-1ba8-46e4-a886-056b366a0fa1-kube-api-access-h6phc\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:06:16.801139 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:16.801079 2571 generic.go:356] "Generic (PLEG): container finished" podID="87fe3dda-1ba8-46e4-a886-056b366a0fa1" containerID="c457a7cbdef514b665a2e827f416791202d92da5206f5904fbea9ac84840e1ea" exitCode=0 Jul 10 19:06:16.801570 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:16.801150 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pcdzp" Jul 10 19:06:16.801570 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:16.801159 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pcdzp" event={"ID":"87fe3dda-1ba8-46e4-a886-056b366a0fa1","Type":"ContainerDied","Data":"c457a7cbdef514b665a2e827f416791202d92da5206f5904fbea9ac84840e1ea"} Jul 10 19:06:16.801570 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:16.801219 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pcdzp" event={"ID":"87fe3dda-1ba8-46e4-a886-056b366a0fa1","Type":"ContainerDied","Data":"ab4067b5715ce3514ba7e39d1fba1d9e7235b1c9a497265bc87526a67c21627b"} Jul 10 19:06:16.801570 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:16.801254 2571 scope.go:117] "RemoveContainer" containerID="c457a7cbdef514b665a2e827f416791202d92da5206f5904fbea9ac84840e1ea" Jul 10 19:06:16.810315 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:16.810298 2571 scope.go:117] "RemoveContainer" containerID="c457a7cbdef514b665a2e827f416791202d92da5206f5904fbea9ac84840e1ea" Jul 10 19:06:16.810556 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:06:16.810538 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c457a7cbdef514b665a2e827f416791202d92da5206f5904fbea9ac84840e1ea\": container with ID starting with c457a7cbdef514b665a2e827f416791202d92da5206f5904fbea9ac84840e1ea not found: ID does not exist" containerID="c457a7cbdef514b665a2e827f416791202d92da5206f5904fbea9ac84840e1ea" Jul 10 19:06:16.810619 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:16.810569 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c457a7cbdef514b665a2e827f416791202d92da5206f5904fbea9ac84840e1ea"} err="failed to get container status \"c457a7cbdef514b665a2e827f416791202d92da5206f5904fbea9ac84840e1ea\": rpc error: code = NotFound desc = could not find container \"c457a7cbdef514b665a2e827f416791202d92da5206f5904fbea9ac84840e1ea\": container with ID starting with c457a7cbdef514b665a2e827f416791202d92da5206f5904fbea9ac84840e1ea not found: ID does not exist" Jul 10 19:06:16.822239 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:16.822211 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pcdzp"] Jul 10 19:06:16.828095 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:16.828076 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pcdzp"] Jul 10 19:06:17.018343 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:17.018310 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87fe3dda-1ba8-46e4-a886-056b366a0fa1" path="/var/lib/kubelet/pods/87fe3dda-1ba8-46e4-a886-056b366a0fa1/volumes" Jul 10 19:06:23.392190 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:23.392157 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-btldc" Jul 10 19:06:23.392621 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:23.392214 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-btldc" Jul 10 19:06:23.414011 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:23.413983 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-btldc" Jul 10 19:06:23.861449 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:23.861421 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-btldc" Jul 10 19:06:45.060936 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:45.060902 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-djx2g"] Jul 10 19:06:45.061420 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:45.061403 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87fe3dda-1ba8-46e4-a886-056b366a0fa1" containerName="registry-server" Jul 10 19:06:45.061465 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:45.061422 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87fe3dda-1ba8-46e4-a886-056b366a0fa1" containerName="registry-server" Jul 10 19:06:45.061503 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:45.061484 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="87fe3dda-1ba8-46e4-a886-056b366a0fa1" containerName="registry-server" Jul 10 19:06:45.063460 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:45.063443 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-djx2g" Jul 10 19:06:45.069347 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:45.069326 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-cns2n\"" Jul 10 19:06:45.078684 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:45.078650 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-djx2g"] Jul 10 19:06:45.162584 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:45.162543 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5swz6\" (UniqueName: \"kubernetes.io/projected/4cb58ef8-03fe-499a-944e-af2e69a86c76-kube-api-access-5swz6\") pod \"authorino-operator-75cc8d5cb7-djx2g\" (UID: \"4cb58ef8-03fe-499a-944e-af2e69a86c76\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-djx2g" Jul 10 19:06:45.264011 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:45.263976 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5swz6\" (UniqueName: \"kubernetes.io/projected/4cb58ef8-03fe-499a-944e-af2e69a86c76-kube-api-access-5swz6\") pod \"authorino-operator-75cc8d5cb7-djx2g\" (UID: \"4cb58ef8-03fe-499a-944e-af2e69a86c76\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-djx2g" Jul 10 19:06:45.272794 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:45.272769 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5swz6\" (UniqueName: \"kubernetes.io/projected/4cb58ef8-03fe-499a-944e-af2e69a86c76-kube-api-access-5swz6\") pod \"authorino-operator-75cc8d5cb7-djx2g\" (UID: \"4cb58ef8-03fe-499a-944e-af2e69a86c76\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-djx2g" Jul 10 19:06:45.374480 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:45.374416 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-djx2g" Jul 10 19:06:45.504967 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:45.504940 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-djx2g"] Jul 10 19:06:45.507621 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:06:45.507589 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb58ef8_03fe_499a_944e_af2e69a86c76.slice/crio-b581daedcd9124ea6cc9b57f75576e5060dc9b1d7a85f529a2346e8318560ff1 WatchSource:0}: Error finding container b581daedcd9124ea6cc9b57f75576e5060dc9b1d7a85f529a2346e8318560ff1: Status 404 returned error can't find the container with id b581daedcd9124ea6cc9b57f75576e5060dc9b1d7a85f529a2346e8318560ff1 Jul 10 19:06:45.936689 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:45.936656 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-djx2g" event={"ID":"4cb58ef8-03fe-499a-944e-af2e69a86c76","Type":"ContainerStarted","Data":"b581daedcd9124ea6cc9b57f75576e5060dc9b1d7a85f529a2346e8318560ff1"} Jul 10 19:06:47.949263 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:47.949131 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-djx2g" event={"ID":"4cb58ef8-03fe-499a-944e-af2e69a86c76","Type":"ContainerStarted","Data":"8ddf203d975d6cb16088176fb3e9f604ae477af1d714ef36a97d1e474bce7cdf"} Jul 10 19:06:47.949263 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:47.949243 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-djx2g" Jul 10 19:06:47.987447 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:47.987399 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-djx2g" podStartSLOduration=0.863083479 podStartE2EDuration="2.987382614s" totalImagesPullingTime="2.124299135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:06:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:06:47.986966045 +0000 UTC m=+389.523308477" watchObservedRunningTime="2026-07-10 19:06:47.987382614 +0000 UTC m=+389.523725047" Jul 10 19:06:54.799540 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:54.799503 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php"] Jul 10 19:06:54.801999 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:54.801982 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" Jul 10 19:06:54.809040 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:54.809021 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zhzck\"" Jul 10 19:06:54.819798 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:54.819768 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php"] Jul 10 19:06:54.852609 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:54.852578 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktgvc\" (UniqueName: \"kubernetes.io/projected/b41c6ce0-426f-471f-bb58-ed85dca90bb4-kube-api-access-ktgvc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-94php\" (UID: \"b41c6ce0-426f-471f-bb58-ed85dca90bb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" Jul 10 19:06:54.852741 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:54.852613 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b41c6ce0-426f-471f-bb58-ed85dca90bb4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-94php\" (UID: \"b41c6ce0-426f-471f-bb58-ed85dca90bb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" Jul 10 19:06:54.953488 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:54.953462 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktgvc\" (UniqueName: \"kubernetes.io/projected/b41c6ce0-426f-471f-bb58-ed85dca90bb4-kube-api-access-ktgvc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-94php\" (UID: \"b41c6ce0-426f-471f-bb58-ed85dca90bb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" Jul 10 19:06:54.953642 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:54.953495 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b41c6ce0-426f-471f-bb58-ed85dca90bb4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-94php\" (UID: \"b41c6ce0-426f-471f-bb58-ed85dca90bb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" Jul 10 19:06:54.953916 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:54.953898 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b41c6ce0-426f-471f-bb58-ed85dca90bb4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-94php\" (UID: \"b41c6ce0-426f-471f-bb58-ed85dca90bb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" Jul 10 19:06:54.962890 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:54.962873 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktgvc\" (UniqueName: \"kubernetes.io/projected/b41c6ce0-426f-471f-bb58-ed85dca90bb4-kube-api-access-ktgvc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-94php\" (UID: \"b41c6ce0-426f-471f-bb58-ed85dca90bb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" Jul 10 19:06:55.112392 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:55.112309 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" Jul 10 19:06:55.249712 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:55.249684 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php"] Jul 10 19:06:55.251682 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:06:55.251657 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb41c6ce0_426f_471f_bb58_ed85dca90bb4.slice/crio-7edd24dd1ec04def1065bf0ea99b4a11fb418087910bbe34f437cdb8d536aafe WatchSource:0}: Error finding container 7edd24dd1ec04def1065bf0ea99b4a11fb418087910bbe34f437cdb8d536aafe: Status 404 returned error can't find the container with id 7edd24dd1ec04def1065bf0ea99b4a11fb418087910bbe34f437cdb8d536aafe Jul 10 19:06:55.992925 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:55.992887 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" event={"ID":"b41c6ce0-426f-471f-bb58-ed85dca90bb4","Type":"ContainerStarted","Data":"7edd24dd1ec04def1065bf0ea99b4a11fb418087910bbe34f437cdb8d536aafe"} Jul 10 19:06:58.955961 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:58.955931 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-djx2g" Jul 10 19:06:59.018613 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:59.018580 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" event={"ID":"b41c6ce0-426f-471f-bb58-ed85dca90bb4","Type":"ContainerStarted","Data":"9216b458fb97d3e550fe39c8ccc89e939eb7ba5e7cec98315ebfd7e62d0777b7"} Jul 10 19:06:59.018772 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:59.018636 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" Jul 10 19:06:59.044963 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:06:59.044909 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" podStartSLOduration=1.76980302 podStartE2EDuration="5.044890127s" totalImagesPullingTime="3.275087107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:06:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:06:59.043638668 +0000 UTC m=+400.579981099" watchObservedRunningTime="2026-07-10 19:06:59.044890127 +0000 UTC m=+400.581232560" Jul 10 19:07:10.024410 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:10.024378 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" Jul 10 19:07:11.618644 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:11.618612 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php"] Jul 10 19:07:11.619013 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:11.618813 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" podUID="b41c6ce0-426f-471f-bb58-ed85dca90bb4" containerName="manager" containerID="cri-o://9216b458fb97d3e550fe39c8ccc89e939eb7ba5e7cec98315ebfd7e62d0777b7" gracePeriod=2 Jul 10 19:07:11.623509 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:11.623479 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php"] Jul 10 19:07:11.636583 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:11.636557 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv"] Jul 10 19:07:11.636986 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:11.636967 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b41c6ce0-426f-471f-bb58-ed85dca90bb4" containerName="manager" Jul 10 19:07:11.637087 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:11.636988 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b41c6ce0-426f-471f-bb58-ed85dca90bb4" containerName="manager" Jul 10 19:07:11.637175 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:11.637124 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b41c6ce0-426f-471f-bb58-ed85dca90bb4" containerName="manager" Jul 10 19:07:11.640969 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:11.640930 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv" Jul 10 19:07:11.643349 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:07:11.643321 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-94php\" is forbidden: User \"system:node:ip-10-0-131-229.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-229.ec2.internal' and this object" podUID="b41c6ce0-426f-471f-bb58-ed85dca90bb4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" Jul 10 19:07:11.662877 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:11.662857 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv"] Jul 10 19:07:11.804630 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:11.804604 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwrh7\" (UniqueName: \"kubernetes.io/projected/c4ee5deb-0b6a-48a9-a3b2-fca777e236b4-kube-api-access-hwrh7\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8x5tv\" (UID: \"c4ee5deb-0b6a-48a9-a3b2-fca777e236b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv" Jul 10 19:07:11.804758 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:11.804735 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4ee5deb-0b6a-48a9-a3b2-fca777e236b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8x5tv\" (UID: \"c4ee5deb-0b6a-48a9-a3b2-fca777e236b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv" Jul 10 19:07:11.860621 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:11.860596 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" Jul 10 19:07:11.862917 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:07:11.862894 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-94php\" is forbidden: User \"system:node:ip-10-0-131-229.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-229.ec2.internal' and this object" podUID="b41c6ce0-426f-471f-bb58-ed85dca90bb4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" Jul 10 19:07:11.905480 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:11.905398 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4ee5deb-0b6a-48a9-a3b2-fca777e236b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8x5tv\" (UID: \"c4ee5deb-0b6a-48a9-a3b2-fca777e236b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv" Jul 10 19:07:11.905480 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:11.905446 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwrh7\" (UniqueName: \"kubernetes.io/projected/c4ee5deb-0b6a-48a9-a3b2-fca777e236b4-kube-api-access-hwrh7\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8x5tv\" (UID: \"c4ee5deb-0b6a-48a9-a3b2-fca777e236b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv" Jul 10 19:07:11.905824 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:11.905801 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4ee5deb-0b6a-48a9-a3b2-fca777e236b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8x5tv\" (UID: \"c4ee5deb-0b6a-48a9-a3b2-fca777e236b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv" Jul 10 19:07:11.914072 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:11.914050 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwrh7\" (UniqueName: \"kubernetes.io/projected/c4ee5deb-0b6a-48a9-a3b2-fca777e236b4-kube-api-access-hwrh7\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8x5tv\" (UID: \"c4ee5deb-0b6a-48a9-a3b2-fca777e236b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv" Jul 10 19:07:11.993808 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:11.993774 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv" Jul 10 19:07:12.006620 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:12.006596 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktgvc\" (UniqueName: \"kubernetes.io/projected/b41c6ce0-426f-471f-bb58-ed85dca90bb4-kube-api-access-ktgvc\") pod \"b41c6ce0-426f-471f-bb58-ed85dca90bb4\" (UID: \"b41c6ce0-426f-471f-bb58-ed85dca90bb4\") " Jul 10 19:07:12.006730 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:12.006672 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b41c6ce0-426f-471f-bb58-ed85dca90bb4-extensions-socket-volume\") pod \"b41c6ce0-426f-471f-bb58-ed85dca90bb4\" (UID: \"b41c6ce0-426f-471f-bb58-ed85dca90bb4\") " Jul 10 19:07:12.007233 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:12.007199 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b41c6ce0-426f-471f-bb58-ed85dca90bb4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b41c6ce0-426f-471f-bb58-ed85dca90bb4" (UID: "b41c6ce0-426f-471f-bb58-ed85dca90bb4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:07:12.008513 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:12.008488 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b41c6ce0-426f-471f-bb58-ed85dca90bb4-kube-api-access-ktgvc" (OuterVolumeSpecName: "kube-api-access-ktgvc") pod "b41c6ce0-426f-471f-bb58-ed85dca90bb4" (UID: "b41c6ce0-426f-471f-bb58-ed85dca90bb4"). InnerVolumeSpecName "kube-api-access-ktgvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:07:12.079738 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:12.079703 2571 generic.go:356] "Generic (PLEG): container finished" podID="b41c6ce0-426f-471f-bb58-ed85dca90bb4" containerID="9216b458fb97d3e550fe39c8ccc89e939eb7ba5e7cec98315ebfd7e62d0777b7" exitCode=0 Jul 10 19:07:12.079870 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:12.079745 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" Jul 10 19:07:12.079870 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:12.079771 2571 scope.go:117] "RemoveContainer" containerID="9216b458fb97d3e550fe39c8ccc89e939eb7ba5e7cec98315ebfd7e62d0777b7" Jul 10 19:07:12.082592 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:07:12.082431 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-94php\" is forbidden: User \"system:node:ip-10-0-131-229.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-229.ec2.internal' and this object" podUID="b41c6ce0-426f-471f-bb58-ed85dca90bb4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" Jul 10 19:07:12.091079 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:12.091056 2571 scope.go:117] "RemoveContainer" containerID="9216b458fb97d3e550fe39c8ccc89e939eb7ba5e7cec98315ebfd7e62d0777b7" Jul 10 19:07:12.091427 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:07:12.091403 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9216b458fb97d3e550fe39c8ccc89e939eb7ba5e7cec98315ebfd7e62d0777b7\": container with ID starting with 9216b458fb97d3e550fe39c8ccc89e939eb7ba5e7cec98315ebfd7e62d0777b7 not found: ID does not exist" containerID="9216b458fb97d3e550fe39c8ccc89e939eb7ba5e7cec98315ebfd7e62d0777b7" Jul 10 19:07:12.091491 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:12.091441 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9216b458fb97d3e550fe39c8ccc89e939eb7ba5e7cec98315ebfd7e62d0777b7"} err="failed to get container status \"9216b458fb97d3e550fe39c8ccc89e939eb7ba5e7cec98315ebfd7e62d0777b7\": rpc error: code = NotFound desc = could not find container \"9216b458fb97d3e550fe39c8ccc89e939eb7ba5e7cec98315ebfd7e62d0777b7\": container with ID starting with 9216b458fb97d3e550fe39c8ccc89e939eb7ba5e7cec98315ebfd7e62d0777b7 not found: ID does not exist" Jul 10 19:07:12.092938 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:07:12.092908 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-94php\" is forbidden: User \"system:node:ip-10-0-131-229.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-229.ec2.internal' and this object" podUID="b41c6ce0-426f-471f-bb58-ed85dca90bb4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-94php" Jul 10 19:07:12.108841 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:12.108617 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktgvc\" (UniqueName: \"kubernetes.io/projected/b41c6ce0-426f-471f-bb58-ed85dca90bb4-kube-api-access-ktgvc\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:07:12.108841 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:12.108643 2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b41c6ce0-426f-471f-bb58-ed85dca90bb4-extensions-socket-volume\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:07:12.142150 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:12.137075 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv"] Jul 10 19:07:13.017522 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:13.017440 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b41c6ce0-426f-471f-bb58-ed85dca90bb4" path="/var/lib/kubelet/pods/b41c6ce0-426f-471f-bb58-ed85dca90bb4/volumes" Jul 10 19:07:13.085910 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:13.085869 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv" event={"ID":"c4ee5deb-0b6a-48a9-a3b2-fca777e236b4","Type":"ContainerStarted","Data":"69cb2b299ac24a197d3f1f53c12e6d5906c734241fc94c2fb0192710953a3016"} Jul 10 19:07:13.086130 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:13.085939 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv" Jul 10 19:07:13.086130 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:13.085965 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv" event={"ID":"c4ee5deb-0b6a-48a9-a3b2-fca777e236b4","Type":"ContainerStarted","Data":"887a9e023df1b9e31cac11c9ac110432efa9f402d0e87f4f4a237f349a1cee9a"} Jul 10 19:07:13.112675 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:13.112624 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv" podStartSLOduration=2.112608566 podStartE2EDuration="2.112608566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:07:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:07:13.109646315 +0000 UTC m=+414.645988748" watchObservedRunningTime="2026-07-10 19:07:13.112608566 +0000 UTC m=+414.648951056" Jul 10 19:07:24.093790 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:24.093756 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv" Jul 10 19:07:36.858453 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:36.858415 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv"] Jul 10 19:07:36.858903 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:36.858625 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv" podUID="c4ee5deb-0b6a-48a9-a3b2-fca777e236b4" containerName="manager" containerID="cri-o://69cb2b299ac24a197d3f1f53c12e6d5906c734241fc94c2fb0192710953a3016" gracePeriod=10 Jul 10 19:07:37.111759 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:37.111698 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv" Jul 10 19:07:37.194776 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:37.194746 2571 generic.go:356] "Generic (PLEG): container finished" podID="c4ee5deb-0b6a-48a9-a3b2-fca777e236b4" containerID="69cb2b299ac24a197d3f1f53c12e6d5906c734241fc94c2fb0192710953a3016" exitCode=0 Jul 10 19:07:37.194905 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:37.194785 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv" Jul 10 19:07:37.194905 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:37.194829 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv" event={"ID":"c4ee5deb-0b6a-48a9-a3b2-fca777e236b4","Type":"ContainerDied","Data":"69cb2b299ac24a197d3f1f53c12e6d5906c734241fc94c2fb0192710953a3016"} Jul 10 19:07:37.194905 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:37.194878 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv" event={"ID":"c4ee5deb-0b6a-48a9-a3b2-fca777e236b4","Type":"ContainerDied","Data":"887a9e023df1b9e31cac11c9ac110432efa9f402d0e87f4f4a237f349a1cee9a"} Jul 10 19:07:37.194905 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:37.194901 2571 scope.go:117] "RemoveContainer" containerID="69cb2b299ac24a197d3f1f53c12e6d5906c734241fc94c2fb0192710953a3016" Jul 10 19:07:37.203436 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:37.203416 2571 scope.go:117] "RemoveContainer" containerID="69cb2b299ac24a197d3f1f53c12e6d5906c734241fc94c2fb0192710953a3016" Jul 10 19:07:37.203712 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:07:37.203689 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69cb2b299ac24a197d3f1f53c12e6d5906c734241fc94c2fb0192710953a3016\": container with ID starting with 69cb2b299ac24a197d3f1f53c12e6d5906c734241fc94c2fb0192710953a3016 not found: ID does not exist" containerID="69cb2b299ac24a197d3f1f53c12e6d5906c734241fc94c2fb0192710953a3016" Jul 10 19:07:37.203760 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:37.203721 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69cb2b299ac24a197d3f1f53c12e6d5906c734241fc94c2fb0192710953a3016"} err="failed to get container status \"69cb2b299ac24a197d3f1f53c12e6d5906c734241fc94c2fb0192710953a3016\": rpc error: code = NotFound desc = could not find container \"69cb2b299ac24a197d3f1f53c12e6d5906c734241fc94c2fb0192710953a3016\": container with ID starting with 69cb2b299ac24a197d3f1f53c12e6d5906c734241fc94c2fb0192710953a3016 not found: ID does not exist" Jul 10 19:07:37.222025 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:37.222000 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4ee5deb-0b6a-48a9-a3b2-fca777e236b4-extensions-socket-volume\") pod \"c4ee5deb-0b6a-48a9-a3b2-fca777e236b4\" (UID: \"c4ee5deb-0b6a-48a9-a3b2-fca777e236b4\") " Jul 10 19:07:37.222158 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:37.222070 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwrh7\" (UniqueName: \"kubernetes.io/projected/c4ee5deb-0b6a-48a9-a3b2-fca777e236b4-kube-api-access-hwrh7\") pod \"c4ee5deb-0b6a-48a9-a3b2-fca777e236b4\" (UID: \"c4ee5deb-0b6a-48a9-a3b2-fca777e236b4\") " Jul 10 19:07:37.222419 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:37.222398 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4ee5deb-0b6a-48a9-a3b2-fca777e236b4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c4ee5deb-0b6a-48a9-a3b2-fca777e236b4" (UID: "c4ee5deb-0b6a-48a9-a3b2-fca777e236b4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:07:37.224080 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:37.224063 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4ee5deb-0b6a-48a9-a3b2-fca777e236b4-kube-api-access-hwrh7" (OuterVolumeSpecName: "kube-api-access-hwrh7") pod "c4ee5deb-0b6a-48a9-a3b2-fca777e236b4" (UID: "c4ee5deb-0b6a-48a9-a3b2-fca777e236b4"). InnerVolumeSpecName "kube-api-access-hwrh7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:07:37.323861 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:37.323830 2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4ee5deb-0b6a-48a9-a3b2-fca777e236b4-extensions-socket-volume\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:07:37.323861 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:37.323858 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwrh7\" (UniqueName: \"kubernetes.io/projected/c4ee5deb-0b6a-48a9-a3b2-fca777e236b4-kube-api-access-hwrh7\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:07:37.523287 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:37.523255 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv"] Jul 10 19:07:37.527076 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:37.527055 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8x5tv"] Jul 10 19:07:39.018323 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:39.018286 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4ee5deb-0b6a-48a9-a3b2-fca777e236b4" path="/var/lib/kubelet/pods/c4ee5deb-0b6a-48a9-a3b2-fca777e236b4/volumes" Jul 10 19:07:40.493241 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.493211 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767"] Jul 10 19:07:40.493616 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.493601 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4ee5deb-0b6a-48a9-a3b2-fca777e236b4" containerName="manager" Jul 10 19:07:40.493662 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.493617 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4ee5deb-0b6a-48a9-a3b2-fca777e236b4" containerName="manager" Jul 10 19:07:40.493697 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.493686 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4ee5deb-0b6a-48a9-a3b2-fca777e236b4" containerName="manager" Jul 10 19:07:40.496605 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.496589 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.499331 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.499308 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 19:07:40.499465 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.499318 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-nfcnh\"" Jul 10 19:07:40.510067 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.510044 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767"] Jul 10 19:07:40.551404 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.551348 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56f8901b-5cec-4f5a-8090-fbc9473159ac-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.551581 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.551448 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56f8901b-5cec-4f5a-8090-fbc9473159ac-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.551581 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.551495 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56f8901b-5cec-4f5a-8090-fbc9473159ac-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.551581 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.551555 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56f8901b-5cec-4f5a-8090-fbc9473159ac-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.551801 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.551581 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56f8901b-5cec-4f5a-8090-fbc9473159ac-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.551801 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.551605 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56f8901b-5cec-4f5a-8090-fbc9473159ac-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.551801 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.551635 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56f8901b-5cec-4f5a-8090-fbc9473159ac-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.551801 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.551670 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl9sg\" (UniqueName: \"kubernetes.io/projected/56f8901b-5cec-4f5a-8090-fbc9473159ac-kube-api-access-dl9sg\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.551801 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.551715 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56f8901b-5cec-4f5a-8090-fbc9473159ac-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.652236 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.652202 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl9sg\" (UniqueName: \"kubernetes.io/projected/56f8901b-5cec-4f5a-8090-fbc9473159ac-kube-api-access-dl9sg\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.652412 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.652253 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56f8901b-5cec-4f5a-8090-fbc9473159ac-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.652412 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.652305 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56f8901b-5cec-4f5a-8090-fbc9473159ac-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.652412 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.652340 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56f8901b-5cec-4f5a-8090-fbc9473159ac-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.652596 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.652533 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56f8901b-5cec-4f5a-8090-fbc9473159ac-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.652653 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.652604 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56f8901b-5cec-4f5a-8090-fbc9473159ac-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.652745 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.652717 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56f8901b-5cec-4f5a-8090-fbc9473159ac-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.652816 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.652764 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56f8901b-5cec-4f5a-8090-fbc9473159ac-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.652816 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.652731 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56f8901b-5cec-4f5a-8090-fbc9473159ac-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.652816 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.652803 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56f8901b-5cec-4f5a-8090-fbc9473159ac-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.652993 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.652775 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56f8901b-5cec-4f5a-8090-fbc9473159ac-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.652993 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.652907 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56f8901b-5cec-4f5a-8090-fbc9473159ac-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.653252 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.653209 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56f8901b-5cec-4f5a-8090-fbc9473159ac-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.653451 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.653433 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56f8901b-5cec-4f5a-8090-fbc9473159ac-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.654687 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.654668 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56f8901b-5cec-4f5a-8090-fbc9473159ac-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.654903 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.654879 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56f8901b-5cec-4f5a-8090-fbc9473159ac-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.661731 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.661705 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56f8901b-5cec-4f5a-8090-fbc9473159ac-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.662024 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.662003 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl9sg\" (UniqueName: \"kubernetes.io/projected/56f8901b-5cec-4f5a-8090-fbc9473159ac-kube-api-access-dl9sg\") pod \"maas-default-gateway-openshift-default-5896d4dd49-v2767\" (UID: \"56f8901b-5cec-4f5a-8090-fbc9473159ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.807038 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.807009 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:40.952483 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.951844 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 19:07:40.952483 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.951926 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 19:07:40.952483 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.951961 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 19:07:40.954922 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:40.954896 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767"] Jul 10 19:07:41.215761 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:41.215651 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" event={"ID":"56f8901b-5cec-4f5a-8090-fbc9473159ac","Type":"ContainerStarted","Data":"a09dd4c330e5cff61f0d9394b5408c2919e7db5c8c1b912c55f99e4c3329cb79"} Jul 10 19:07:41.215761 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:41.215709 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" event={"ID":"56f8901b-5cec-4f5a-8090-fbc9473159ac","Type":"ContainerStarted","Data":"7d07554249e7c799bed3986ee50ecf2a5165741d5dcd522eebd443a4a0cb1f10"} Jul 10 19:07:41.240486 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:41.240438 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" podStartSLOduration=1.240422387 podStartE2EDuration="1.240422387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:07:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:07:41.238318 +0000 UTC m=+442.774660435" watchObservedRunningTime="2026-07-10 19:07:41.240422387 +0000 UTC m=+442.776764818" Jul 10 19:07:41.807342 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:41.807308 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:42.812694 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:42.812665 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:43.224822 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:43.224726 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:43.225862 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:43.225840 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:07:55.836815 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:55.836781 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z82l7"] Jul 10 19:07:55.839232 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:55.839215 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z82l7" Jul 10 19:07:55.841939 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:55.841917 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 10 19:07:55.842073 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:55.842018 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-k59z6\"" Jul 10 19:07:55.850134 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:55.850114 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z82l7"] Jul 10 19:07:55.931812 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:55.931778 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z82l7"] Jul 10 19:07:55.987763 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:55.987731 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmgkm\" (UniqueName: \"kubernetes.io/projected/de902065-8382-435a-bd09-207ad54d01b5-kube-api-access-wmgkm\") pod \"limitador-limitador-64c94fd64f-z82l7\" (UID: \"de902065-8382-435a-bd09-207ad54d01b5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z82l7" Jul 10 19:07:55.987886 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:55.987778 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/de902065-8382-435a-bd09-207ad54d01b5-config-file\") pod \"limitador-limitador-64c94fd64f-z82l7\" (UID: \"de902065-8382-435a-bd09-207ad54d01b5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z82l7" Jul 10 19:07:56.088862 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:56.088781 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgkm\" (UniqueName: \"kubernetes.io/projected/de902065-8382-435a-bd09-207ad54d01b5-kube-api-access-wmgkm\") pod \"limitador-limitador-64c94fd64f-z82l7\" (UID: \"de902065-8382-435a-bd09-207ad54d01b5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z82l7" Jul 10 19:07:56.088862 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:56.088846 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/de902065-8382-435a-bd09-207ad54d01b5-config-file\") pod \"limitador-limitador-64c94fd64f-z82l7\" (UID: \"de902065-8382-435a-bd09-207ad54d01b5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z82l7" Jul 10 19:07:56.089486 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:56.089467 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/de902065-8382-435a-bd09-207ad54d01b5-config-file\") pod \"limitador-limitador-64c94fd64f-z82l7\" (UID: \"de902065-8382-435a-bd09-207ad54d01b5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z82l7" Jul 10 19:07:56.097649 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:56.097624 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmgkm\" (UniqueName: \"kubernetes.io/projected/de902065-8382-435a-bd09-207ad54d01b5-kube-api-access-wmgkm\") pod \"limitador-limitador-64c94fd64f-z82l7\" (UID: \"de902065-8382-435a-bd09-207ad54d01b5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z82l7" Jul 10 19:07:56.151007 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:56.150978 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z82l7" Jul 10 19:07:56.295941 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:56.295918 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z82l7"] Jul 10 19:07:56.297969 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:07:56.297940 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde902065_8382_435a_bd09_207ad54d01b5.slice/crio-c97e490b5a4f6a59cdb7d956cb4489187598a34b0f848775ef715354d6b299a9 WatchSource:0}: Error finding container c97e490b5a4f6a59cdb7d956cb4489187598a34b0f848775ef715354d6b299a9: Status 404 returned error can't find the container with id c97e490b5a4f6a59cdb7d956cb4489187598a34b0f848775ef715354d6b299a9 Jul 10 19:07:56.650340 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:56.650313 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-4z6zj"] Jul 10 19:07:56.653329 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:56.653302 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4z6zj" Jul 10 19:07:56.656168 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:56.656134 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jdvs4\"" Jul 10 19:07:56.661054 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:56.660949 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-4z6zj"] Jul 10 19:07:56.797531 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:56.797484 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2krj\" (UniqueName: \"kubernetes.io/projected/cf920758-84dc-4914-9e09-a48595430ff3-kube-api-access-q2krj\") pod \"authorino-58cd955675-4z6zj\" (UID: \"cf920758-84dc-4914-9e09-a48595430ff3\") " pod="kuadrant-system/authorino-58cd955675-4z6zj" Jul 10 19:07:56.899140 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:56.899054 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2krj\" (UniqueName: \"kubernetes.io/projected/cf920758-84dc-4914-9e09-a48595430ff3-kube-api-access-q2krj\") pod \"authorino-58cd955675-4z6zj\" (UID: \"cf920758-84dc-4914-9e09-a48595430ff3\") " pod="kuadrant-system/authorino-58cd955675-4z6zj" Jul 10 19:07:56.911255 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:56.911166 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2krj\" (UniqueName: \"kubernetes.io/projected/cf920758-84dc-4914-9e09-a48595430ff3-kube-api-access-q2krj\") pod \"authorino-58cd955675-4z6zj\" (UID: \"cf920758-84dc-4914-9e09-a48595430ff3\") " pod="kuadrant-system/authorino-58cd955675-4z6zj" Jul 10 19:07:56.964330 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:56.964285 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4z6zj" Jul 10 19:07:57.140432 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:57.140405 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-4z6zj"] Jul 10 19:07:57.143145 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:07:57.143112 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf920758_84dc_4914_9e09_a48595430ff3.slice/crio-ebc6bda68760d72246b38c9a242950212d877548ddbb406cfb24a09ac7eac775 WatchSource:0}: Error finding container ebc6bda68760d72246b38c9a242950212d877548ddbb406cfb24a09ac7eac775: Status 404 returned error can't find the container with id ebc6bda68760d72246b38c9a242950212d877548ddbb406cfb24a09ac7eac775 Jul 10 19:07:57.291042 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:57.291007 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4z6zj" event={"ID":"cf920758-84dc-4914-9e09-a48595430ff3","Type":"ContainerStarted","Data":"ebc6bda68760d72246b38c9a242950212d877548ddbb406cfb24a09ac7eac775"} Jul 10 19:07:57.292404 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:07:57.292376 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z82l7" event={"ID":"de902065-8382-435a-bd09-207ad54d01b5","Type":"ContainerStarted","Data":"c97e490b5a4f6a59cdb7d956cb4489187598a34b0f848775ef715354d6b299a9"} Jul 10 19:08:01.006465 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:01.006384 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-4z6zj"] Jul 10 19:08:01.316540 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:01.316502 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z82l7" event={"ID":"de902065-8382-435a-bd09-207ad54d01b5","Type":"ContainerStarted","Data":"091b2ef54b75d170bfcea8ad3d19a757ea9fafa7baf50ff8c43ad56208421dfc"} Jul 10 19:08:01.316798 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:01.316566 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-z82l7" Jul 10 19:08:01.317978 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:01.317949 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4z6zj" event={"ID":"cf920758-84dc-4914-9e09-a48595430ff3","Type":"ContainerStarted","Data":"794e72941f3d4c05302f11ff2f43e63962dcba825953818fa15ad629be189255"} Jul 10 19:08:01.334381 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:01.334337 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-z82l7" podStartSLOduration=2.009316809 podStartE2EDuration="6.334323013s" totalImagesPullingTime="4.325006204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:07:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:01.33314426 +0000 UTC m=+462.869486689" watchObservedRunningTime="2026-07-10 19:08:01.334323013 +0000 UTC m=+462.870665451" Jul 10 19:08:01.348548 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:01.348507 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-4z6zj" podStartSLOduration=1.862240115 podStartE2EDuration="5.348496691s" totalImagesPullingTime="3.486256576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:07:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:01.348080666 +0000 UTC m=+462.884423101" watchObservedRunningTime="2026-07-10 19:08:01.348496691 +0000 UTC m=+462.884839123" Jul 10 19:08:02.322846 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:02.322802 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-4z6zj" podUID="cf920758-84dc-4914-9e09-a48595430ff3" containerName="authorino" containerID="cri-o://794e72941f3d4c05302f11ff2f43e63962dcba825953818fa15ad629be189255" gracePeriod=30 Jul 10 19:08:02.575914 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:02.575861 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4z6zj" Jul 10 19:08:02.752783 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:02.752753 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2krj\" (UniqueName: \"kubernetes.io/projected/cf920758-84dc-4914-9e09-a48595430ff3-kube-api-access-q2krj\") pod \"cf920758-84dc-4914-9e09-a48595430ff3\" (UID: \"cf920758-84dc-4914-9e09-a48595430ff3\") " Jul 10 19:08:02.754605 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:02.754574 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf920758-84dc-4914-9e09-a48595430ff3-kube-api-access-q2krj" (OuterVolumeSpecName: "kube-api-access-q2krj") pod "cf920758-84dc-4914-9e09-a48595430ff3" (UID: "cf920758-84dc-4914-9e09-a48595430ff3"). InnerVolumeSpecName "kube-api-access-q2krj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:08:02.853874 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:02.853788 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2krj\" (UniqueName: \"kubernetes.io/projected/cf920758-84dc-4914-9e09-a48595430ff3-kube-api-access-q2krj\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:08:03.328194 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:03.328165 2571 generic.go:356] "Generic (PLEG): container finished" podID="cf920758-84dc-4914-9e09-a48595430ff3" containerID="794e72941f3d4c05302f11ff2f43e63962dcba825953818fa15ad629be189255" exitCode=0 Jul 10 19:08:03.328616 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:03.328206 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4z6zj" Jul 10 19:08:03.328616 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:03.328236 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4z6zj" event={"ID":"cf920758-84dc-4914-9e09-a48595430ff3","Type":"ContainerDied","Data":"794e72941f3d4c05302f11ff2f43e63962dcba825953818fa15ad629be189255"} Jul 10 19:08:03.328616 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:03.328266 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4z6zj" event={"ID":"cf920758-84dc-4914-9e09-a48595430ff3","Type":"ContainerDied","Data":"ebc6bda68760d72246b38c9a242950212d877548ddbb406cfb24a09ac7eac775"} Jul 10 19:08:03.328616 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:03.328282 2571 scope.go:117] "RemoveContainer" containerID="794e72941f3d4c05302f11ff2f43e63962dcba825953818fa15ad629be189255" Jul 10 19:08:03.336906 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:03.336890 2571 scope.go:117] "RemoveContainer" containerID="794e72941f3d4c05302f11ff2f43e63962dcba825953818fa15ad629be189255" Jul 10 19:08:03.337173 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:08:03.337156 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"794e72941f3d4c05302f11ff2f43e63962dcba825953818fa15ad629be189255\": container with ID starting with 794e72941f3d4c05302f11ff2f43e63962dcba825953818fa15ad629be189255 not found: ID does not exist" containerID="794e72941f3d4c05302f11ff2f43e63962dcba825953818fa15ad629be189255" Jul 10 19:08:03.337227 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:03.337182 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"794e72941f3d4c05302f11ff2f43e63962dcba825953818fa15ad629be189255"} err="failed to get container status \"794e72941f3d4c05302f11ff2f43e63962dcba825953818fa15ad629be189255\": rpc error: code = NotFound desc = could not find container \"794e72941f3d4c05302f11ff2f43e63962dcba825953818fa15ad629be189255\": container with ID starting with 794e72941f3d4c05302f11ff2f43e63962dcba825953818fa15ad629be189255 not found: ID does not exist" Jul 10 19:08:03.346996 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:03.346970 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-4z6zj"] Jul 10 19:08:03.351557 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:03.351537 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-4z6zj"] Jul 10 19:08:05.018319 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:05.018279 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf920758-84dc-4914-9e09-a48595430ff3" path="/var/lib/kubelet/pods/cf920758-84dc-4914-9e09-a48595430ff3/volumes" Jul 10 19:08:11.019640 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:11.019600 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z82l7"] Jul 10 19:08:11.020127 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:11.019827 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-z82l7" podUID="de902065-8382-435a-bd09-207ad54d01b5" containerName="limitador" containerID="cri-o://091b2ef54b75d170bfcea8ad3d19a757ea9fafa7baf50ff8c43ad56208421dfc" gracePeriod=30 Jul 10 19:08:11.020410 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:11.020387 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-z82l7" Jul 10 19:08:11.576041 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:11.576019 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z82l7" Jul 10 19:08:11.630015 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:11.629957 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmgkm\" (UniqueName: \"kubernetes.io/projected/de902065-8382-435a-bd09-207ad54d01b5-kube-api-access-wmgkm\") pod \"de902065-8382-435a-bd09-207ad54d01b5\" (UID: \"de902065-8382-435a-bd09-207ad54d01b5\") " Jul 10 19:08:11.630158 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:11.630046 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/de902065-8382-435a-bd09-207ad54d01b5-config-file\") pod \"de902065-8382-435a-bd09-207ad54d01b5\" (UID: \"de902065-8382-435a-bd09-207ad54d01b5\") " Jul 10 19:08:11.630385 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:11.630361 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de902065-8382-435a-bd09-207ad54d01b5-config-file" (OuterVolumeSpecName: "config-file") pod "de902065-8382-435a-bd09-207ad54d01b5" (UID: "de902065-8382-435a-bd09-207ad54d01b5"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:08:11.631881 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:11.631861 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de902065-8382-435a-bd09-207ad54d01b5-kube-api-access-wmgkm" (OuterVolumeSpecName: "kube-api-access-wmgkm") pod "de902065-8382-435a-bd09-207ad54d01b5" (UID: "de902065-8382-435a-bd09-207ad54d01b5"). InnerVolumeSpecName "kube-api-access-wmgkm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:08:11.731154 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:11.731127 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmgkm\" (UniqueName: \"kubernetes.io/projected/de902065-8382-435a-bd09-207ad54d01b5-kube-api-access-wmgkm\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:08:11.731154 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:11.731151 2571 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/de902065-8382-435a-bd09-207ad54d01b5-config-file\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:08:12.373985 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:12.373947 2571 generic.go:356] "Generic (PLEG): container finished" podID="de902065-8382-435a-bd09-207ad54d01b5" containerID="091b2ef54b75d170bfcea8ad3d19a757ea9fafa7baf50ff8c43ad56208421dfc" exitCode=0 Jul 10 19:08:12.374462 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:12.373992 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z82l7" Jul 10 19:08:12.374462 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:12.374025 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z82l7" event={"ID":"de902065-8382-435a-bd09-207ad54d01b5","Type":"ContainerDied","Data":"091b2ef54b75d170bfcea8ad3d19a757ea9fafa7baf50ff8c43ad56208421dfc"} Jul 10 19:08:12.374462 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:12.374076 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z82l7" event={"ID":"de902065-8382-435a-bd09-207ad54d01b5","Type":"ContainerDied","Data":"c97e490b5a4f6a59cdb7d956cb4489187598a34b0f848775ef715354d6b299a9"} Jul 10 19:08:12.374462 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:12.374120 2571 scope.go:117] "RemoveContainer" containerID="091b2ef54b75d170bfcea8ad3d19a757ea9fafa7baf50ff8c43ad56208421dfc" Jul 10 19:08:12.382895 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:12.382877 2571 scope.go:117] "RemoveContainer" containerID="091b2ef54b75d170bfcea8ad3d19a757ea9fafa7baf50ff8c43ad56208421dfc" Jul 10 19:08:12.383176 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:08:12.383158 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"091b2ef54b75d170bfcea8ad3d19a757ea9fafa7baf50ff8c43ad56208421dfc\": container with ID starting with 091b2ef54b75d170bfcea8ad3d19a757ea9fafa7baf50ff8c43ad56208421dfc not found: ID does not exist" containerID="091b2ef54b75d170bfcea8ad3d19a757ea9fafa7baf50ff8c43ad56208421dfc" Jul 10 19:08:12.383254 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:12.383185 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"091b2ef54b75d170bfcea8ad3d19a757ea9fafa7baf50ff8c43ad56208421dfc"} err="failed to get container status \"091b2ef54b75d170bfcea8ad3d19a757ea9fafa7baf50ff8c43ad56208421dfc\": rpc error: code = NotFound desc = could not find container \"091b2ef54b75d170bfcea8ad3d19a757ea9fafa7baf50ff8c43ad56208421dfc\": container with ID starting with 091b2ef54b75d170bfcea8ad3d19a757ea9fafa7baf50ff8c43ad56208421dfc not found: ID does not exist" Jul 10 19:08:12.396000 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:12.395977 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z82l7"] Jul 10 19:08:12.400053 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:12.400035 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z82l7"] Jul 10 19:08:13.017251 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:13.017219 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de902065-8382-435a-bd09-207ad54d01b5" path="/var/lib/kubelet/pods/de902065-8382-435a-bd09-207ad54d01b5/volumes" Jul 10 19:08:14.156402 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.156370 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wsffq"] Jul 10 19:08:14.156767 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.156755 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de902065-8382-435a-bd09-207ad54d01b5" containerName="limitador" Jul 10 19:08:14.156767 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.156769 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="de902065-8382-435a-bd09-207ad54d01b5" containerName="limitador" Jul 10 19:08:14.156840 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.156782 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf920758-84dc-4914-9e09-a48595430ff3" containerName="authorino" Jul 10 19:08:14.156840 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.156789 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf920758-84dc-4914-9e09-a48595430ff3" containerName="authorino" Jul 10 19:08:14.156900 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.156848 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="de902065-8382-435a-bd09-207ad54d01b5" containerName="limitador" Jul 10 19:08:14.156900 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.156860 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf920758-84dc-4914-9e09-a48595430ff3" containerName="authorino" Jul 10 19:08:14.159936 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.159921 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wsffq" Jul 10 19:08:14.164518 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.164491 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 10 19:08:14.164666 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.164529 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 19:08:14.164666 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.164544 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 10 19:08:14.164666 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.164632 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-v5tcn\"" Jul 10 19:08:14.177381 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.177357 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wsffq"] Jul 10 19:08:14.250565 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.250536 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6l58\" (UniqueName: \"kubernetes.io/projected/97650629-5712-4285-a7e5-3f7107cdb9bc-kube-api-access-j6l58\") pod \"postgres-69fb4594cf-wsffq\" (UID: \"97650629-5712-4285-a7e5-3f7107cdb9bc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wsffq" Jul 10 19:08:14.250695 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.250613 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/97650629-5712-4285-a7e5-3f7107cdb9bc-data\") pod \"postgres-69fb4594cf-wsffq\" (UID: \"97650629-5712-4285-a7e5-3f7107cdb9bc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wsffq" Jul 10 19:08:14.351574 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.351543 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/97650629-5712-4285-a7e5-3f7107cdb9bc-data\") pod \"postgres-69fb4594cf-wsffq\" (UID: \"97650629-5712-4285-a7e5-3f7107cdb9bc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wsffq" Jul 10 19:08:14.351700 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.351679 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6l58\" (UniqueName: \"kubernetes.io/projected/97650629-5712-4285-a7e5-3f7107cdb9bc-kube-api-access-j6l58\") pod \"postgres-69fb4594cf-wsffq\" (UID: \"97650629-5712-4285-a7e5-3f7107cdb9bc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wsffq" Jul 10 19:08:14.351894 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.351876 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/97650629-5712-4285-a7e5-3f7107cdb9bc-data\") pod \"postgres-69fb4594cf-wsffq\" (UID: \"97650629-5712-4285-a7e5-3f7107cdb9bc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wsffq" Jul 10 19:08:14.360205 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.360176 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6l58\" (UniqueName: \"kubernetes.io/projected/97650629-5712-4285-a7e5-3f7107cdb9bc-kube-api-access-j6l58\") pod \"postgres-69fb4594cf-wsffq\" (UID: \"97650629-5712-4285-a7e5-3f7107cdb9bc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wsffq" Jul 10 19:08:14.469653 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.469595 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wsffq" Jul 10 19:08:14.608012 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:14.602854 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wsffq"] Jul 10 19:08:14.609763 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:08:14.609734 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97650629_5712_4285_a7e5_3f7107cdb9bc.slice/crio-18bf554f94608f5928f9e0ddee09cc1809d01bcaae7d4edf145484ba357cd8e6 WatchSource:0}: Error finding container 18bf554f94608f5928f9e0ddee09cc1809d01bcaae7d4edf145484ba357cd8e6: Status 404 returned error can't find the container with id 18bf554f94608f5928f9e0ddee09cc1809d01bcaae7d4edf145484ba357cd8e6 Jul 10 19:08:15.394867 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:15.394820 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wsffq" event={"ID":"97650629-5712-4285-a7e5-3f7107cdb9bc","Type":"ContainerStarted","Data":"18bf554f94608f5928f9e0ddee09cc1809d01bcaae7d4edf145484ba357cd8e6"} Jul 10 19:08:19.407796 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:19.407769 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 10 19:08:20.425397 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:20.425348 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wsffq" event={"ID":"97650629-5712-4285-a7e5-3f7107cdb9bc","Type":"ContainerStarted","Data":"a95377ad899ab6b16c1cd8ae295132d1f5c59a6c0b3eb124aea3774968022fdb"} Jul 10 19:08:20.425775 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:20.425493 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wsffq" Jul 10 19:08:20.445909 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:20.445866 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wsffq" podStartSLOduration=1.651828405 podStartE2EDuration="6.445851926s" totalImagesPullingTime="4.794023521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:20.443523898 +0000 UTC m=+481.979866331" watchObservedRunningTime="2026-07-10 19:08:20.445851926 +0000 UTC m=+481.982194359" Jul 10 19:08:26.458891 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:26.458864 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wsffq" Jul 10 19:08:27.047887 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.047853 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sxm85"] Jul 10 19:08:27.053638 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.053614 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sxm85" Jul 10 19:08:27.057093 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.057077 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jdvs4\"" Jul 10 19:08:27.061468 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.061441 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sxm85"] Jul 10 19:08:27.179617 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.179587 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkcpn\" (UniqueName: \"kubernetes.io/projected/83143ac6-9ff8-4b39-9c69-c7cf89e159b2-kube-api-access-vkcpn\") pod \"authorino-5d4b67c446-sxm85\" (UID: \"83143ac6-9ff8-4b39-9c69-c7cf89e159b2\") " pod="kuadrant-system/authorino-5d4b67c446-sxm85" Jul 10 19:08:27.280553 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.280517 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkcpn\" (UniqueName: \"kubernetes.io/projected/83143ac6-9ff8-4b39-9c69-c7cf89e159b2-kube-api-access-vkcpn\") pod \"authorino-5d4b67c446-sxm85\" (UID: \"83143ac6-9ff8-4b39-9c69-c7cf89e159b2\") " pod="kuadrant-system/authorino-5d4b67c446-sxm85" Jul 10 19:08:27.285492 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.285464 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sxm85"] Jul 10 19:08:27.285779 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:08:27.285756 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-vkcpn], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-sxm85" podUID="83143ac6-9ff8-4b39-9c69-c7cf89e159b2" Jul 10 19:08:27.290977 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.290952 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkcpn\" (UniqueName: \"kubernetes.io/projected/83143ac6-9ff8-4b39-9c69-c7cf89e159b2-kube-api-access-vkcpn\") pod \"authorino-5d4b67c446-sxm85\" (UID: \"83143ac6-9ff8-4b39-9c69-c7cf89e159b2\") " pod="kuadrant-system/authorino-5d4b67c446-sxm85" Jul 10 19:08:27.457084 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.457003 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sxm85" Jul 10 19:08:27.464492 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.464469 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sxm85" Jul 10 19:08:27.583531 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.583503 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkcpn\" (UniqueName: \"kubernetes.io/projected/83143ac6-9ff8-4b39-9c69-c7cf89e159b2-kube-api-access-vkcpn\") pod \"83143ac6-9ff8-4b39-9c69-c7cf89e159b2\" (UID: \"83143ac6-9ff8-4b39-9c69-c7cf89e159b2\") " Jul 10 19:08:27.585541 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.585507 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83143ac6-9ff8-4b39-9c69-c7cf89e159b2-kube-api-access-vkcpn" (OuterVolumeSpecName: "kube-api-access-vkcpn") pod "83143ac6-9ff8-4b39-9c69-c7cf89e159b2" (UID: "83143ac6-9ff8-4b39-9c69-c7cf89e159b2"). InnerVolumeSpecName "kube-api-access-vkcpn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:08:27.592531 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.592506 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-799f8ccb6b-dnmg5"] Jul 10 19:08:27.596488 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.596471 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-799f8ccb6b-dnmg5" Jul 10 19:08:27.599364 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.599343 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 10 19:08:27.606652 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.606628 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-799f8ccb6b-dnmg5"] Jul 10 19:08:27.684936 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.684910 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmxtc\" (UniqueName: \"kubernetes.io/projected/b80f0d86-401b-4b45-898d-ed9484c2325e-kube-api-access-gmxtc\") pod \"authorino-799f8ccb6b-dnmg5\" (UID: \"b80f0d86-401b-4b45-898d-ed9484c2325e\") " pod="kuadrant-system/authorino-799f8ccb6b-dnmg5" Jul 10 19:08:27.685075 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.684956 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b80f0d86-401b-4b45-898d-ed9484c2325e-tls-cert\") pod \"authorino-799f8ccb6b-dnmg5\" (UID: \"b80f0d86-401b-4b45-898d-ed9484c2325e\") " pod="kuadrant-system/authorino-799f8ccb6b-dnmg5" Jul 10 19:08:27.685075 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.684995 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkcpn\" (UniqueName: \"kubernetes.io/projected/83143ac6-9ff8-4b39-9c69-c7cf89e159b2-kube-api-access-vkcpn\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:08:27.785707 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.785673 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmxtc\" (UniqueName: \"kubernetes.io/projected/b80f0d86-401b-4b45-898d-ed9484c2325e-kube-api-access-gmxtc\") pod \"authorino-799f8ccb6b-dnmg5\" (UID: \"b80f0d86-401b-4b45-898d-ed9484c2325e\") " pod="kuadrant-system/authorino-799f8ccb6b-dnmg5" Jul 10 19:08:27.785892 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.785743 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b80f0d86-401b-4b45-898d-ed9484c2325e-tls-cert\") pod \"authorino-799f8ccb6b-dnmg5\" (UID: \"b80f0d86-401b-4b45-898d-ed9484c2325e\") " pod="kuadrant-system/authorino-799f8ccb6b-dnmg5" Jul 10 19:08:27.788272 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.788250 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b80f0d86-401b-4b45-898d-ed9484c2325e-tls-cert\") pod \"authorino-799f8ccb6b-dnmg5\" (UID: \"b80f0d86-401b-4b45-898d-ed9484c2325e\") " pod="kuadrant-system/authorino-799f8ccb6b-dnmg5" Jul 10 19:08:27.795765 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.795736 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmxtc\" (UniqueName: \"kubernetes.io/projected/b80f0d86-401b-4b45-898d-ed9484c2325e-kube-api-access-gmxtc\") pod \"authorino-799f8ccb6b-dnmg5\" (UID: \"b80f0d86-401b-4b45-898d-ed9484c2325e\") " pod="kuadrant-system/authorino-799f8ccb6b-dnmg5" Jul 10 19:08:27.905207 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:27.905178 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-799f8ccb6b-dnmg5" Jul 10 19:08:28.057027 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:28.057003 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-799f8ccb6b-dnmg5"] Jul 10 19:08:28.058343 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:08:28.058316 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb80f0d86_401b_4b45_898d_ed9484c2325e.slice/crio-f00897ee560312b15ed6c4006be9de4c1f2923bad5a66da59b9b0df657bb32e8 WatchSource:0}: Error finding container f00897ee560312b15ed6c4006be9de4c1f2923bad5a66da59b9b0df657bb32e8: Status 404 returned error can't find the container with id f00897ee560312b15ed6c4006be9de4c1f2923bad5a66da59b9b0df657bb32e8 Jul 10 19:08:28.463322 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:28.463245 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sxm85" Jul 10 19:08:28.463322 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:28.463273 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-799f8ccb6b-dnmg5" event={"ID":"b80f0d86-401b-4b45-898d-ed9484c2325e","Type":"ContainerStarted","Data":"f00897ee560312b15ed6c4006be9de4c1f2923bad5a66da59b9b0df657bb32e8"} Jul 10 19:08:28.526468 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:28.526441 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sxm85"] Jul 10 19:08:28.528303 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:28.528285 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sxm85"] Jul 10 19:08:29.018917 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:29.018887 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83143ac6-9ff8-4b39-9c69-c7cf89e159b2" path="/var/lib/kubelet/pods/83143ac6-9ff8-4b39-9c69-c7cf89e159b2/volumes" Jul 10 19:08:29.469286 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:29.469252 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-799f8ccb6b-dnmg5" event={"ID":"b80f0d86-401b-4b45-898d-ed9484c2325e","Type":"ContainerStarted","Data":"26cf05c9d0cfd2014b2a46ff063445fcada2f681c2bb47d0c399f132db0aefd6"} Jul 10 19:08:29.488110 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:08:29.488050 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-799f8ccb6b-dnmg5" podStartSLOduration=2.027844089 podStartE2EDuration="2.488036227s" totalImagesPullingTime="460.192138ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:08:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:29.486198608 +0000 UTC m=+491.022541040" watchObservedRunningTime="2026-07-10 19:08:29.488036227 +0000 UTC m=+491.024378689" Jul 10 19:09:17.694205 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:17.694173 2571 generic.go:356] "Generic (PLEG): container finished" podID="56f8901b-5cec-4f5a-8090-fbc9473159ac" containerID="a09dd4c330e5cff61f0d9394b5408c2919e7db5c8c1b912c55f99e4c3329cb79" exitCode=0 Jul 10 19:09:17.694676 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:17.694246 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" event={"ID":"56f8901b-5cec-4f5a-8090-fbc9473159ac","Type":"ContainerDied","Data":"a09dd4c330e5cff61f0d9394b5408c2919e7db5c8c1b912c55f99e4c3329cb79"} Jul 10 19:09:17.694776 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:17.694759 2571 scope.go:117] "RemoveContainer" containerID="a09dd4c330e5cff61f0d9394b5408c2919e7db5c8c1b912c55f99e4c3329cb79" Jul 10 19:09:17.695777 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:17.695749 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 19:09:17.695876 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:17.695817 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 19:09:17.695876 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:17.695850 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 19:09:18.700485 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:18.700452 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" event={"ID":"56f8901b-5cec-4f5a-8090-fbc9473159ac","Type":"ContainerStarted","Data":"74a27a24f811a50c5ffa8ecafb644e217e74cf8b847b592b75365559aae943e8"} Jul 10 19:09:18.807728 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:18.807692 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:09:18.807977 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:18.807955 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-v2767 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 10 19:09:18.808078 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:18.808007 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" podUID="56f8901b-5cec-4f5a-8090-fbc9473159ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 10 19:09:19.808382 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:19.808342 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-v2767 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 10 19:09:19.808770 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:19.808403 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" podUID="56f8901b-5cec-4f5a-8090-fbc9473159ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 10 19:09:20.808464 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:20.808421 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-v2767 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 10 19:09:20.808880 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:20.808484 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" podUID="56f8901b-5cec-4f5a-8090-fbc9473159ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 10 19:09:21.808035 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:21.807998 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-v2767 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 10 19:09:21.808249 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:21.808058 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" podUID="56f8901b-5cec-4f5a-8090-fbc9473159ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 10 19:09:22.807991 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:22.807952 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-v2767 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 10 19:09:22.808400 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:22.808014 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" podUID="56f8901b-5cec-4f5a-8090-fbc9473159ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 10 19:09:23.811750 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:23.811717 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:09:23.812223 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:23.811988 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:09:23.812702 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:23.812684 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-v2767" Jul 10 19:09:30.702682 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.702650 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh"] Jul 10 19:09:30.705441 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.705422 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:30.714324 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.714296 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 19:09:30.714726 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.714298 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 19:09:30.715495 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.715463 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vc5dn\"" Jul 10 19:09:30.715495 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.715491 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 10 19:09:30.719192 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.718859 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh"] Jul 10 19:09:30.861476 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.861429 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xll6v\" (UniqueName: \"kubernetes.io/projected/9a476776-c067-49f1-aeeb-e6619e36239d-kube-api-access-xll6v\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh\" (UID: \"9a476776-c067-49f1-aeeb-e6619e36239d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:30.861671 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.861517 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a476776-c067-49f1-aeeb-e6619e36239d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh\" (UID: \"9a476776-c067-49f1-aeeb-e6619e36239d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:30.861671 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.861559 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a476776-c067-49f1-aeeb-e6619e36239d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh\" (UID: \"9a476776-c067-49f1-aeeb-e6619e36239d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:30.861671 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.861579 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a476776-c067-49f1-aeeb-e6619e36239d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh\" (UID: \"9a476776-c067-49f1-aeeb-e6619e36239d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:30.861671 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.861641 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a476776-c067-49f1-aeeb-e6619e36239d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh\" (UID: \"9a476776-c067-49f1-aeeb-e6619e36239d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:30.861810 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.861682 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a476776-c067-49f1-aeeb-e6619e36239d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh\" (UID: \"9a476776-c067-49f1-aeeb-e6619e36239d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:30.963075 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.962975 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a476776-c067-49f1-aeeb-e6619e36239d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh\" (UID: \"9a476776-c067-49f1-aeeb-e6619e36239d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:30.963075 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.963056 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a476776-c067-49f1-aeeb-e6619e36239d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh\" (UID: \"9a476776-c067-49f1-aeeb-e6619e36239d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:30.963075 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.963081 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a476776-c067-49f1-aeeb-e6619e36239d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh\" (UID: \"9a476776-c067-49f1-aeeb-e6619e36239d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:30.963365 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.963162 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a476776-c067-49f1-aeeb-e6619e36239d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh\" (UID: \"9a476776-c067-49f1-aeeb-e6619e36239d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:30.963365 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.963191 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a476776-c067-49f1-aeeb-e6619e36239d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh\" (UID: \"9a476776-c067-49f1-aeeb-e6619e36239d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:30.963365 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.963261 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xll6v\" (UniqueName: \"kubernetes.io/projected/9a476776-c067-49f1-aeeb-e6619e36239d-kube-api-access-xll6v\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh\" (UID: \"9a476776-c067-49f1-aeeb-e6619e36239d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:30.963590 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.963563 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a476776-c067-49f1-aeeb-e6619e36239d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh\" (UID: \"9a476776-c067-49f1-aeeb-e6619e36239d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:30.963590 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.963577 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a476776-c067-49f1-aeeb-e6619e36239d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh\" (UID: \"9a476776-c067-49f1-aeeb-e6619e36239d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:30.963688 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.963601 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a476776-c067-49f1-aeeb-e6619e36239d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh\" (UID: \"9a476776-c067-49f1-aeeb-e6619e36239d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:30.965866 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.965836 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a476776-c067-49f1-aeeb-e6619e36239d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh\" (UID: \"9a476776-c067-49f1-aeeb-e6619e36239d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:30.966132 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.966094 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a476776-c067-49f1-aeeb-e6619e36239d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh\" (UID: \"9a476776-c067-49f1-aeeb-e6619e36239d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:30.973361 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:30.973338 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xll6v\" (UniqueName: \"kubernetes.io/projected/9a476776-c067-49f1-aeeb-e6619e36239d-kube-api-access-xll6v\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh\" (UID: \"9a476776-c067-49f1-aeeb-e6619e36239d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:31.022877 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:31.022845 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:31.159617 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:31.159590 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh"] Jul 10 19:09:31.162742 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:09:31.162709 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a476776_c067_49f1_aeeb_e6619e36239d.slice/crio-7d15494d0b5abb070914974e1d8d1bcd23f79dbd55d349e0306076aaf07b84d7 WatchSource:0}: Error finding container 7d15494d0b5abb070914974e1d8d1bcd23f79dbd55d349e0306076aaf07b84d7: Status 404 returned error can't find the container with id 7d15494d0b5abb070914974e1d8d1bcd23f79dbd55d349e0306076aaf07b84d7 Jul 10 19:09:31.768600 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:31.768559 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" event={"ID":"9a476776-c067-49f1-aeeb-e6619e36239d","Type":"ContainerStarted","Data":"7d15494d0b5abb070914974e1d8d1bcd23f79dbd55d349e0306076aaf07b84d7"} Jul 10 19:09:32.774969 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:32.774877 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" event={"ID":"9a476776-c067-49f1-aeeb-e6619e36239d","Type":"ContainerStarted","Data":"64d64fcbfaa324dbf507664168a39b7de3a3411ab2a6391f034612da68f79861"} Jul 10 19:09:32.796254 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:32.796205 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" podStartSLOduration=1.666593097 podStartE2EDuration="2.796187846s" totalImagesPullingTime="1.129594749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:09:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:09:32.795022333 +0000 UTC m=+554.331364757" watchObservedRunningTime="2026-07-10 19:09:32.796187846 +0000 UTC m=+554.332530283" Jul 10 19:09:36.773807 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.773769 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r"] Jul 10 19:09:36.776777 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.776758 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:36.779919 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.779896 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 10 19:09:36.791263 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.791240 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r"] Jul 10 19:09:36.815730 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.815702 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57d5ded1-3ce5-4d9d-a518-264c8254bbcc-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sjq7r\" (UID: \"57d5ded1-3ce5-4d9d-a518-264c8254bbcc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:36.815867 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.815765 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57d5ded1-3ce5-4d9d-a518-264c8254bbcc-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sjq7r\" (UID: \"57d5ded1-3ce5-4d9d-a518-264c8254bbcc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:36.815867 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.815837 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57d5ded1-3ce5-4d9d-a518-264c8254bbcc-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sjq7r\" (UID: \"57d5ded1-3ce5-4d9d-a518-264c8254bbcc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:36.815946 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.815871 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57d5ded1-3ce5-4d9d-a518-264c8254bbcc-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sjq7r\" (UID: \"57d5ded1-3ce5-4d9d-a518-264c8254bbcc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:36.815946 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.815888 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57d5ded1-3ce5-4d9d-a518-264c8254bbcc-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sjq7r\" (UID: \"57d5ded1-3ce5-4d9d-a518-264c8254bbcc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:36.816022 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.815959 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2x6k\" (UniqueName: \"kubernetes.io/projected/57d5ded1-3ce5-4d9d-a518-264c8254bbcc-kube-api-access-q2x6k\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sjq7r\" (UID: \"57d5ded1-3ce5-4d9d-a518-264c8254bbcc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:36.917150 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.917094 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57d5ded1-3ce5-4d9d-a518-264c8254bbcc-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sjq7r\" (UID: \"57d5ded1-3ce5-4d9d-a518-264c8254bbcc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:36.917150 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.917150 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57d5ded1-3ce5-4d9d-a518-264c8254bbcc-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sjq7r\" (UID: \"57d5ded1-3ce5-4d9d-a518-264c8254bbcc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:36.917381 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.917181 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2x6k\" (UniqueName: \"kubernetes.io/projected/57d5ded1-3ce5-4d9d-a518-264c8254bbcc-kube-api-access-q2x6k\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sjq7r\" (UID: \"57d5ded1-3ce5-4d9d-a518-264c8254bbcc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:36.917381 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.917205 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57d5ded1-3ce5-4d9d-a518-264c8254bbcc-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sjq7r\" (UID: \"57d5ded1-3ce5-4d9d-a518-264c8254bbcc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:36.917518 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.917408 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57d5ded1-3ce5-4d9d-a518-264c8254bbcc-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sjq7r\" (UID: \"57d5ded1-3ce5-4d9d-a518-264c8254bbcc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:36.917518 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.917466 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57d5ded1-3ce5-4d9d-a518-264c8254bbcc-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sjq7r\" (UID: \"57d5ded1-3ce5-4d9d-a518-264c8254bbcc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:36.917633 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.917568 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57d5ded1-3ce5-4d9d-a518-264c8254bbcc-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sjq7r\" (UID: \"57d5ded1-3ce5-4d9d-a518-264c8254bbcc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:36.917670 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.917659 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57d5ded1-3ce5-4d9d-a518-264c8254bbcc-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sjq7r\" (UID: \"57d5ded1-3ce5-4d9d-a518-264c8254bbcc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:36.917841 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.917815 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57d5ded1-3ce5-4d9d-a518-264c8254bbcc-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sjq7r\" (UID: \"57d5ded1-3ce5-4d9d-a518-264c8254bbcc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:36.919455 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.919439 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57d5ded1-3ce5-4d9d-a518-264c8254bbcc-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sjq7r\" (UID: \"57d5ded1-3ce5-4d9d-a518-264c8254bbcc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:36.919624 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.919606 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57d5ded1-3ce5-4d9d-a518-264c8254bbcc-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sjq7r\" (UID: \"57d5ded1-3ce5-4d9d-a518-264c8254bbcc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:36.926455 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:36.926428 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2x6k\" (UniqueName: \"kubernetes.io/projected/57d5ded1-3ce5-4d9d-a518-264c8254bbcc-kube-api-access-q2x6k\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sjq7r\" (UID: \"57d5ded1-3ce5-4d9d-a518-264c8254bbcc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:37.087565 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:37.087533 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:37.217857 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:37.217834 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r"] Jul 10 19:09:37.219749 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:09:37.219717 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57d5ded1_3ce5_4d9d_a518_264c8254bbcc.slice/crio-8f5179c4f1d4428d894d075f3a78e464ffc345730a190966e5a4f6a6f4f56f7a WatchSource:0}: Error finding container 8f5179c4f1d4428d894d075f3a78e464ffc345730a190966e5a4f6a6f4f56f7a: Status 404 returned error can't find the container with id 8f5179c4f1d4428d894d075f3a78e464ffc345730a190966e5a4f6a6f4f56f7a Jul 10 19:09:37.802610 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:37.802567 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" event={"ID":"57d5ded1-3ce5-4d9d-a518-264c8254bbcc","Type":"ContainerStarted","Data":"b4acc72c6adcf7358a9e86abd0c668363e960aba2875ba28c969ce5ebd9abc94"} Jul 10 19:09:37.802975 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:37.802641 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" event={"ID":"57d5ded1-3ce5-4d9d-a518-264c8254bbcc","Type":"ContainerStarted","Data":"8f5179c4f1d4428d894d075f3a78e464ffc345730a190966e5a4f6a6f4f56f7a"} Jul 10 19:09:37.822778 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:37.822730 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" podStartSLOduration=1.6376135330000001 podStartE2EDuration="1.822700283s" totalImagesPullingTime="185.08675ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:09:37.821783301 +0000 UTC m=+559.358125735" watchObservedRunningTime="2026-07-10 19:09:37.822700283 +0000 UTC m=+559.359042715" Jul 10 19:09:41.022953 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:41.022915 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:41.023388 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:41.022971 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:41.035571 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:41.035550 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:41.833478 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:41.833452 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh" Jul 10 19:09:47.087772 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:47.087739 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:47.088250 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:47.087801 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:47.099970 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:47.099948 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:09:47.864230 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:09:47.864200 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sjq7r" Jul 10 19:10:10.770377 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:10.770342 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk"] Jul 10 19:10:10.774288 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:10.774259 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:10.777164 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:10.777144 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 10 19:10:10.784514 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:10.784490 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk"] Jul 10 19:10:10.940554 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:10.940514 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb08f413-7474-4391-b673-60a52eac1d7e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk\" (UID: \"eb08f413-7474-4391-b673-60a52eac1d7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:10.940554 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:10.940560 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb08f413-7474-4391-b673-60a52eac1d7e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk\" (UID: \"eb08f413-7474-4391-b673-60a52eac1d7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:10.940763 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:10.940597 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb08f413-7474-4391-b673-60a52eac1d7e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk\" (UID: \"eb08f413-7474-4391-b673-60a52eac1d7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:10.940763 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:10.940625 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb08f413-7474-4391-b673-60a52eac1d7e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk\" (UID: \"eb08f413-7474-4391-b673-60a52eac1d7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:10.940763 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:10.940653 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb08f413-7474-4391-b673-60a52eac1d7e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk\" (UID: \"eb08f413-7474-4391-b673-60a52eac1d7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:10.940763 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:10.940724 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmkw9\" (UniqueName: \"kubernetes.io/projected/eb08f413-7474-4391-b673-60a52eac1d7e-kube-api-access-mmkw9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk\" (UID: \"eb08f413-7474-4391-b673-60a52eac1d7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:11.041706 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:11.041678 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb08f413-7474-4391-b673-60a52eac1d7e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk\" (UID: \"eb08f413-7474-4391-b673-60a52eac1d7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:11.041863 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:11.041709 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb08f413-7474-4391-b673-60a52eac1d7e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk\" (UID: \"eb08f413-7474-4391-b673-60a52eac1d7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:11.041863 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:11.041741 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb08f413-7474-4391-b673-60a52eac1d7e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk\" (UID: \"eb08f413-7474-4391-b673-60a52eac1d7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:11.041863 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:11.041777 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmkw9\" (UniqueName: \"kubernetes.io/projected/eb08f413-7474-4391-b673-60a52eac1d7e-kube-api-access-mmkw9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk\" (UID: \"eb08f413-7474-4391-b673-60a52eac1d7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:11.041863 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:11.041829 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb08f413-7474-4391-b673-60a52eac1d7e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk\" (UID: \"eb08f413-7474-4391-b673-60a52eac1d7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:11.041863 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:11.041855 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb08f413-7474-4391-b673-60a52eac1d7e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk\" (UID: \"eb08f413-7474-4391-b673-60a52eac1d7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:11.042199 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:11.042173 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb08f413-7474-4391-b673-60a52eac1d7e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk\" (UID: \"eb08f413-7474-4391-b673-60a52eac1d7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:11.042296 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:11.042211 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb08f413-7474-4391-b673-60a52eac1d7e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk\" (UID: \"eb08f413-7474-4391-b673-60a52eac1d7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:11.042296 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:11.042255 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb08f413-7474-4391-b673-60a52eac1d7e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk\" (UID: \"eb08f413-7474-4391-b673-60a52eac1d7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:11.044207 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:11.044184 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb08f413-7474-4391-b673-60a52eac1d7e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk\" (UID: \"eb08f413-7474-4391-b673-60a52eac1d7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:11.044415 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:11.044401 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb08f413-7474-4391-b673-60a52eac1d7e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk\" (UID: \"eb08f413-7474-4391-b673-60a52eac1d7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:11.052495 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:11.052473 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmkw9\" (UniqueName: \"kubernetes.io/projected/eb08f413-7474-4391-b673-60a52eac1d7e-kube-api-access-mmkw9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk\" (UID: \"eb08f413-7474-4391-b673-60a52eac1d7e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:11.086219 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:11.086193 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:11.224315 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:11.224290 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk"] Jul 10 19:10:11.226052 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:10:11.226020 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb08f413_7474_4391_b673_60a52eac1d7e.slice/crio-4e9b0b5be5634b9756ada79b7eaad6dda817e45224b318f8d5acfc92ff0a11ac WatchSource:0}: Error finding container 4e9b0b5be5634b9756ada79b7eaad6dda817e45224b318f8d5acfc92ff0a11ac: Status 404 returned error can't find the container with id 4e9b0b5be5634b9756ada79b7eaad6dda817e45224b318f8d5acfc92ff0a11ac Jul 10 19:10:11.969418 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:11.969379 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" event={"ID":"eb08f413-7474-4391-b673-60a52eac1d7e","Type":"ContainerStarted","Data":"fec119797ae5ddb0732d90856f6335c6510df6058daf2424a7bc900db3be1200"} Jul 10 19:10:11.969800 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:11.969436 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" event={"ID":"eb08f413-7474-4391-b673-60a52eac1d7e","Type":"ContainerStarted","Data":"4e9b0b5be5634b9756ada79b7eaad6dda817e45224b318f8d5acfc92ff0a11ac"} Jul 10 19:10:11.991324 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:11.991280 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" podStartSLOduration=1.795191511 podStartE2EDuration="1.99126443s" totalImagesPullingTime="196.072919ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:10:11.989829756 +0000 UTC m=+593.526172190" watchObservedRunningTime="2026-07-10 19:10:11.99126443 +0000 UTC m=+593.527606861" Jul 10 19:10:18.934918 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:18.934880 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:10:18.944259 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:18.944236 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:10:21.086989 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:21.086941 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:21.087402 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:21.087013 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:21.099633 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:21.099610 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:10:22.031772 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:10:22.031746 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk" Jul 10 19:11:47.130030 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:47.129985 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-799f8ccb6b-dnmg5"] Jul 10 19:11:47.130654 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:47.130192 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-799f8ccb6b-dnmg5" podUID="b80f0d86-401b-4b45-898d-ed9484c2325e" containerName="authorino" containerID="cri-o://26cf05c9d0cfd2014b2a46ff063445fcada2f681c2bb47d0c399f132db0aefd6" gracePeriod=30 Jul 10 19:11:47.381302 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:47.381248 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-799f8ccb6b-dnmg5" Jul 10 19:11:47.417495 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:47.417459 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmxtc\" (UniqueName: \"kubernetes.io/projected/b80f0d86-401b-4b45-898d-ed9484c2325e-kube-api-access-gmxtc\") pod \"b80f0d86-401b-4b45-898d-ed9484c2325e\" (UID: \"b80f0d86-401b-4b45-898d-ed9484c2325e\") " Jul 10 19:11:47.417633 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:47.417620 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b80f0d86-401b-4b45-898d-ed9484c2325e-tls-cert\") pod \"b80f0d86-401b-4b45-898d-ed9484c2325e\" (UID: \"b80f0d86-401b-4b45-898d-ed9484c2325e\") " Jul 10 19:11:47.419556 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:47.419525 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b80f0d86-401b-4b45-898d-ed9484c2325e-kube-api-access-gmxtc" (OuterVolumeSpecName: "kube-api-access-gmxtc") pod "b80f0d86-401b-4b45-898d-ed9484c2325e" (UID: "b80f0d86-401b-4b45-898d-ed9484c2325e"). InnerVolumeSpecName "kube-api-access-gmxtc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:11:47.428587 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:47.428555 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b80f0d86-401b-4b45-898d-ed9484c2325e-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "b80f0d86-401b-4b45-898d-ed9484c2325e" (UID: "b80f0d86-401b-4b45-898d-ed9484c2325e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:11:47.438251 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:47.438222 2571 generic.go:356] "Generic (PLEG): container finished" podID="b80f0d86-401b-4b45-898d-ed9484c2325e" containerID="26cf05c9d0cfd2014b2a46ff063445fcada2f681c2bb47d0c399f132db0aefd6" exitCode=0 Jul 10 19:11:47.438354 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:47.438255 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-799f8ccb6b-dnmg5" Jul 10 19:11:47.438354 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:47.438304 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-799f8ccb6b-dnmg5" event={"ID":"b80f0d86-401b-4b45-898d-ed9484c2325e","Type":"ContainerDied","Data":"26cf05c9d0cfd2014b2a46ff063445fcada2f681c2bb47d0c399f132db0aefd6"} Jul 10 19:11:47.438471 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:47.438353 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-799f8ccb6b-dnmg5" event={"ID":"b80f0d86-401b-4b45-898d-ed9484c2325e","Type":"ContainerDied","Data":"f00897ee560312b15ed6c4006be9de4c1f2923bad5a66da59b9b0df657bb32e8"} Jul 10 19:11:47.438471 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:47.438376 2571 scope.go:117] "RemoveContainer" containerID="26cf05c9d0cfd2014b2a46ff063445fcada2f681c2bb47d0c399f132db0aefd6" Jul 10 19:11:47.448356 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:47.448339 2571 scope.go:117] "RemoveContainer" containerID="26cf05c9d0cfd2014b2a46ff063445fcada2f681c2bb47d0c399f132db0aefd6" Jul 10 19:11:47.448624 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:11:47.448606 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26cf05c9d0cfd2014b2a46ff063445fcada2f681c2bb47d0c399f132db0aefd6\": container with ID starting with 26cf05c9d0cfd2014b2a46ff063445fcada2f681c2bb47d0c399f132db0aefd6 not found: ID does not exist" containerID="26cf05c9d0cfd2014b2a46ff063445fcada2f681c2bb47d0c399f132db0aefd6" Jul 10 19:11:47.448679 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:47.448631 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26cf05c9d0cfd2014b2a46ff063445fcada2f681c2bb47d0c399f132db0aefd6"} err="failed to get container status \"26cf05c9d0cfd2014b2a46ff063445fcada2f681c2bb47d0c399f132db0aefd6\": rpc error: code = NotFound desc = could not find container \"26cf05c9d0cfd2014b2a46ff063445fcada2f681c2bb47d0c399f132db0aefd6\": container with ID starting with 26cf05c9d0cfd2014b2a46ff063445fcada2f681c2bb47d0c399f132db0aefd6 not found: ID does not exist" Jul 10 19:11:47.485538 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:47.485513 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-799f8ccb6b-dnmg5"] Jul 10 19:11:47.496367 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:47.496344 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-799f8ccb6b-dnmg5"] Jul 10 19:11:47.518928 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:47.518889 2571 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b80f0d86-401b-4b45-898d-ed9484c2325e-tls-cert\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:11:47.518928 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:47.518925 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmxtc\" (UniqueName: \"kubernetes.io/projected/b80f0d86-401b-4b45-898d-ed9484c2325e-kube-api-access-gmxtc\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:11:49.019657 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:11:49.019625 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b80f0d86-401b-4b45-898d-ed9484c2325e" path="/var/lib/kubelet/pods/b80f0d86-401b-4b45-898d-ed9484c2325e/volumes" Jul 10 19:15:00.152451 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:15:00.152419 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728515-2tkk8"] Jul 10 19:15:00.154003 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:15:00.152824 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b80f0d86-401b-4b45-898d-ed9484c2325e" containerName="authorino" Jul 10 19:15:00.154003 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:15:00.152835 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b80f0d86-401b-4b45-898d-ed9484c2325e" containerName="authorino" Jul 10 19:15:00.154003 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:15:00.152899 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b80f0d86-401b-4b45-898d-ed9484c2325e" containerName="authorino" Jul 10 19:15:00.154688 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:15:00.154664 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728515-2tkk8" Jul 10 19:15:00.157502 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:15:00.157479 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ngc6z\"" Jul 10 19:15:00.164371 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:15:00.164353 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728515-2tkk8" Jul 10 19:15:00.177042 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:15:00.177016 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728515-2tkk8"] Jul 10 19:15:00.301288 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:15:00.301263 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728515-2tkk8"] Jul 10 19:15:00.303141 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:15:00.303089 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb167a208_0f3f_4b68_9412_1a1b2d265642.slice/crio-48afa572d05d6ef72c763ea31df6ae885e70da3056052579a78384796000ceba WatchSource:0}: Error finding container 48afa572d05d6ef72c763ea31df6ae885e70da3056052579a78384796000ceba: Status 404 returned error can't find the container with id 48afa572d05d6ef72c763ea31df6ae885e70da3056052579a78384796000ceba Jul 10 19:15:00.304944 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:15:00.304929 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 19:15:00.362868 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:15:00.362831 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728515-2tkk8" event={"ID":"b167a208-0f3f-4b68-9412-1a1b2d265642","Type":"ContainerStarted","Data":"48afa572d05d6ef72c763ea31df6ae885e70da3056052579a78384796000ceba"} Jul 10 19:15:04.384394 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:15:04.384308 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728515-2tkk8" event={"ID":"b167a208-0f3f-4b68-9412-1a1b2d265642","Type":"ContainerStarted","Data":"42945e9915675690ea71aea5352a761fe9cbf2953aa104536a04572e0f85ee35"} Jul 10 19:15:04.403284 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:15:04.403244 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728515-2tkk8" podStartSLOduration=0.969810407 podStartE2EDuration="4.403229237s" totalImagesPullingTime="3.43341883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:15:04.401558993 +0000 UTC m=+885.937901425" watchObservedRunningTime="2026-07-10 19:15:04.403229237 +0000 UTC m=+885.939571669" Jul 10 19:15:18.976954 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:15:18.976882 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:15:18.977409 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:15:18.976976 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:16:00.011810 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:16:00.011774 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728515-2tkk8"] Jul 10 19:16:00.012355 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:16:00.011998 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728515-2tkk8" podUID="b167a208-0f3f-4b68-9412-1a1b2d265642" containerName="cleanup" containerID="cri-o://42945e9915675690ea71aea5352a761fe9cbf2953aa104536a04572e0f85ee35" gracePeriod=30 Jul 10 19:16:30.200937 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:16:30.200915 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728515-2tkk8" Jul 10 19:16:30.800952 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:16:30.800915 2571 generic.go:356] "Generic (PLEG): container finished" podID="b167a208-0f3f-4b68-9412-1a1b2d265642" containerID="42945e9915675690ea71aea5352a761fe9cbf2953aa104536a04572e0f85ee35" exitCode=137 Jul 10 19:16:30.801156 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:16:30.800959 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728515-2tkk8" Jul 10 19:16:30.801156 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:16:30.801009 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728515-2tkk8" event={"ID":"b167a208-0f3f-4b68-9412-1a1b2d265642","Type":"ContainerDied","Data":"42945e9915675690ea71aea5352a761fe9cbf2953aa104536a04572e0f85ee35"} Jul 10 19:16:30.801156 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:16:30.801066 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728515-2tkk8" event={"ID":"b167a208-0f3f-4b68-9412-1a1b2d265642","Type":"ContainerDied","Data":"48afa572d05d6ef72c763ea31df6ae885e70da3056052579a78384796000ceba"} Jul 10 19:16:30.801156 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:16:30.801090 2571 scope.go:117] "RemoveContainer" containerID="42945e9915675690ea71aea5352a761fe9cbf2953aa104536a04572e0f85ee35" Jul 10 19:16:30.810037 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:16:30.810021 2571 scope.go:117] "RemoveContainer" containerID="42945e9915675690ea71aea5352a761fe9cbf2953aa104536a04572e0f85ee35" Jul 10 19:16:30.810307 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:16:30.810291 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42945e9915675690ea71aea5352a761fe9cbf2953aa104536a04572e0f85ee35\": container with ID starting with 42945e9915675690ea71aea5352a761fe9cbf2953aa104536a04572e0f85ee35 not found: ID does not exist" containerID="42945e9915675690ea71aea5352a761fe9cbf2953aa104536a04572e0f85ee35" Jul 10 19:16:30.810356 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:16:30.810315 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42945e9915675690ea71aea5352a761fe9cbf2953aa104536a04572e0f85ee35"} err="failed to get container status \"42945e9915675690ea71aea5352a761fe9cbf2953aa104536a04572e0f85ee35\": rpc error: code = NotFound desc = could not find container \"42945e9915675690ea71aea5352a761fe9cbf2953aa104536a04572e0f85ee35\": container with ID starting with 42945e9915675690ea71aea5352a761fe9cbf2953aa104536a04572e0f85ee35 not found: ID does not exist" Jul 10 19:16:30.822448 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:16:30.822424 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728515-2tkk8"] Jul 10 19:16:30.824264 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:16:30.824244 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728515-2tkk8"] Jul 10 19:16:31.018597 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:16:31.018570 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b167a208-0f3f-4b68-9412-1a1b2d265642" path="/var/lib/kubelet/pods/b167a208-0f3f-4b68-9412-1a1b2d265642/volumes" Jul 10 19:20:19.015771 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:20:19.015742 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:20:19.017340 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:20:19.016547 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:25:19.045806 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:25:19.045776 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:25:19.048856 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:25:19.048826 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:30:00.140527 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:30:00.140493 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728530-wx8c9"] Jul 10 19:30:00.141116 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:30:00.141081 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b167a208-0f3f-4b68-9412-1a1b2d265642" containerName="cleanup" Jul 10 19:30:00.141205 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:30:00.141121 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b167a208-0f3f-4b68-9412-1a1b2d265642" containerName="cleanup" Jul 10 19:30:00.141279 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:30:00.141263 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b167a208-0f3f-4b68-9412-1a1b2d265642" containerName="cleanup" Jul 10 19:30:00.144392 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:30:00.144370 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-wx8c9" Jul 10 19:30:00.147119 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:30:00.147082 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ngc6z\"" Jul 10 19:30:00.151605 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:30:00.151415 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728530-wx8c9"] Jul 10 19:30:00.155561 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:30:00.155541 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-wx8c9" Jul 10 19:30:00.289772 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:30:00.289745 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728530-wx8c9"] Jul 10 19:30:00.292256 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:30:00.292230 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9bbf5ee_e2fb_410a_a0e7_ff792974ac60.slice/crio-29c100c4d08148dbf6549d359151569d2fe40b720ed8ac60ce1a0e049318ad60 WatchSource:0}: Error finding container 29c100c4d08148dbf6549d359151569d2fe40b720ed8ac60ce1a0e049318ad60: Status 404 returned error can't find the container with id 29c100c4d08148dbf6549d359151569d2fe40b720ed8ac60ce1a0e049318ad60 Jul 10 19:30:00.703172 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:30:00.703136 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-wx8c9" event={"ID":"e9bbf5ee-e2fb-410a-a0e7-ff792974ac60","Type":"ContainerStarted","Data":"7e28df6b620289a680000c9e0e2a82ad32765f4fe30b1592fdf37ae64f639e25"} Jul 10 19:30:00.703327 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:30:00.703194 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-wx8c9" event={"ID":"e9bbf5ee-e2fb-410a-a0e7-ff792974ac60","Type":"ContainerStarted","Data":"29c100c4d08148dbf6549d359151569d2fe40b720ed8ac60ce1a0e049318ad60"} Jul 10 19:30:00.720363 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:30:00.720314 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-wx8c9" podStartSLOduration=0.720300125 podStartE2EDuration="720.300125ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:30:00.719182575 +0000 UTC m=+1782.255525011" watchObservedRunningTime="2026-07-10 19:30:00.720300125 +0000 UTC m=+1782.256642575" Jul 10 19:30:19.077990 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:30:19.077876 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:30:19.092149 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:30:19.082760 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:31:00.011494 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:00.011460 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728530-wx8c9"] Jul 10 19:31:00.012028 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:00.011644 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-wx8c9" podUID="e9bbf5ee-e2fb-410a-a0e7-ff792974ac60" containerName="cleanup" containerID="cri-o://7e28df6b620289a680000c9e0e2a82ad32765f4fe30b1592fdf37ae64f639e25" gracePeriod=30 Jul 10 19:31:18.112778 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:18.112738 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd"] Jul 10 19:31:18.116786 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:18.116763 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" Jul 10 19:31:18.119564 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:18.119542 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pj288\"" Jul 10 19:31:18.119684 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:18.119545 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-69b0c1-serving-cert\"" Jul 10 19:31:18.127147 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:18.127121 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd"] Jul 10 19:31:18.220714 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:18.220682 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73542c15-b3aa-449d-a2cd-a07456be5e2e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd\" (UID: \"73542c15-b3aa-449d-a2cd-a07456be5e2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" Jul 10 19:31:18.220855 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:18.220734 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73542c15-b3aa-449d-a2cd-a07456be5e2e-maas-api-tls\") pod \"maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd\" (UID: \"73542c15-b3aa-449d-a2cd-a07456be5e2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" Jul 10 19:31:18.220855 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:18.220787 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7fmt\" (UniqueName: \"kubernetes.io/projected/73542c15-b3aa-449d-a2cd-a07456be5e2e-kube-api-access-d7fmt\") pod \"maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd\" (UID: \"73542c15-b3aa-449d-a2cd-a07456be5e2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" Jul 10 19:31:18.322154 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:18.322091 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73542c15-b3aa-449d-a2cd-a07456be5e2e-maas-api-tls\") pod \"maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd\" (UID: \"73542c15-b3aa-449d-a2cd-a07456be5e2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" Jul 10 19:31:18.322154 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:18.322159 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7fmt\" (UniqueName: \"kubernetes.io/projected/73542c15-b3aa-449d-a2cd-a07456be5e2e-kube-api-access-d7fmt\") pod \"maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd\" (UID: \"73542c15-b3aa-449d-a2cd-a07456be5e2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" Jul 10 19:31:18.322404 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:18.322258 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73542c15-b3aa-449d-a2cd-a07456be5e2e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd\" (UID: \"73542c15-b3aa-449d-a2cd-a07456be5e2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" Jul 10 19:31:18.322819 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:18.322796 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73542c15-b3aa-449d-a2cd-a07456be5e2e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd\" (UID: \"73542c15-b3aa-449d-a2cd-a07456be5e2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" Jul 10 19:31:18.324615 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:18.324585 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73542c15-b3aa-449d-a2cd-a07456be5e2e-maas-api-tls\") pod \"maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd\" (UID: \"73542c15-b3aa-449d-a2cd-a07456be5e2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" Jul 10 19:31:18.332082 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:18.332059 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7fmt\" (UniqueName: \"kubernetes.io/projected/73542c15-b3aa-449d-a2cd-a07456be5e2e-kube-api-access-d7fmt\") pod \"maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd\" (UID: \"73542c15-b3aa-449d-a2cd-a07456be5e2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" Jul 10 19:31:18.427723 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:18.427669 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" Jul 10 19:31:18.594833 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:18.594765 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd"] Jul 10 19:31:18.597107 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:31:18.597068 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73542c15_b3aa_449d_a2cd_a07456be5e2e.slice/crio-7eded7c8f3632df239db14e5a370398edd82b62515f5aba08ad7d2e11b1ef63f WatchSource:0}: Error finding container 7eded7c8f3632df239db14e5a370398edd82b62515f5aba08ad7d2e11b1ef63f: Status 404 returned error can't find the container with id 7eded7c8f3632df239db14e5a370398edd82b62515f5aba08ad7d2e11b1ef63f Jul 10 19:31:18.598501 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:18.598483 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 19:31:19.097637 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:19.097606 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" event={"ID":"73542c15-b3aa-449d-a2cd-a07456be5e2e","Type":"ContainerStarted","Data":"7eded7c8f3632df239db14e5a370398edd82b62515f5aba08ad7d2e11b1ef63f"} Jul 10 19:31:21.109397 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:21.109363 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" event={"ID":"73542c15-b3aa-449d-a2cd-a07456be5e2e","Type":"ContainerStarted","Data":"ddccdd2f653ca3333c71ee02e96b5f91b17c60e322ba2efc032422ca21c85924"} Jul 10 19:31:21.109797 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:21.109443 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" Jul 10 19:31:21.132554 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:21.132509 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" podStartSLOduration=1.5172362609999999 podStartE2EDuration="3.132496393s" totalImagesPullingTime="1.615260132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:31:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:31:21.130074102 +0000 UTC m=+1862.666416534" watchObservedRunningTime="2026-07-10 19:31:21.132496393 +0000 UTC m=+1862.668838825" Jul 10 19:31:27.120175 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:27.120147 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" Jul 10 19:31:30.156209 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:30.156179 2571 generic.go:356] "Generic (PLEG): container finished" podID="e9bbf5ee-e2fb-410a-a0e7-ff792974ac60" containerID="7e28df6b620289a680000c9e0e2a82ad32765f4fe30b1592fdf37ae64f639e25" exitCode=137 Jul 10 19:31:30.156549 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:30.156221 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-wx8c9" event={"ID":"e9bbf5ee-e2fb-410a-a0e7-ff792974ac60","Type":"ContainerDied","Data":"7e28df6b620289a680000c9e0e2a82ad32765f4fe30b1592fdf37ae64f639e25"} Jul 10 19:31:30.185327 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:30.185302 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-wx8c9" Jul 10 19:31:31.164293 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:31.164249 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-wx8c9" Jul 10 19:31:31.164831 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:31.164618 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-wx8c9" event={"ID":"e9bbf5ee-e2fb-410a-a0e7-ff792974ac60","Type":"ContainerDied","Data":"29c100c4d08148dbf6549d359151569d2fe40b720ed8ac60ce1a0e049318ad60"} Jul 10 19:31:31.164959 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:31.164874 2571 scope.go:117] "RemoveContainer" containerID="7e28df6b620289a680000c9e0e2a82ad32765f4fe30b1592fdf37ae64f639e25" Jul 10 19:31:31.184714 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:31.184692 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728530-wx8c9"] Jul 10 19:31:31.188645 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:31.188623 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728530-wx8c9"] Jul 10 19:31:33.019427 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:31:33.019386 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9bbf5ee-e2fb-410a-a0e7-ff792974ac60" path="/var/lib/kubelet/pods/e9bbf5ee-e2fb-410a-a0e7-ff792974ac60/volumes" Jul 10 19:33:11.233077 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.232984 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd"] Jul 10 19:33:11.233648 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.233275 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" podUID="73542c15-b3aa-449d-a2cd-a07456be5e2e" containerName="maas-api" containerID="cri-o://ddccdd2f653ca3333c71ee02e96b5f91b17c60e322ba2efc032422ca21c85924" gracePeriod=30 Jul 10 19:33:11.484793 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.484734 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" Jul 10 19:33:11.566985 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.566957 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73542c15-b3aa-449d-a2cd-a07456be5e2e-openshift-service-ca\") pod \"73542c15-b3aa-449d-a2cd-a07456be5e2e\" (UID: \"73542c15-b3aa-449d-a2cd-a07456be5e2e\") " Jul 10 19:33:11.567152 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.566995 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73542c15-b3aa-449d-a2cd-a07456be5e2e-maas-api-tls\") pod \"73542c15-b3aa-449d-a2cd-a07456be5e2e\" (UID: \"73542c15-b3aa-449d-a2cd-a07456be5e2e\") " Jul 10 19:33:11.567152 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.567136 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7fmt\" (UniqueName: \"kubernetes.io/projected/73542c15-b3aa-449d-a2cd-a07456be5e2e-kube-api-access-d7fmt\") pod \"73542c15-b3aa-449d-a2cd-a07456be5e2e\" (UID: \"73542c15-b3aa-449d-a2cd-a07456be5e2e\") " Jul 10 19:33:11.567316 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.567293 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73542c15-b3aa-449d-a2cd-a07456be5e2e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "73542c15-b3aa-449d-a2cd-a07456be5e2e" (UID: "73542c15-b3aa-449d-a2cd-a07456be5e2e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:33:11.567480 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.567456 2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73542c15-b3aa-449d-a2cd-a07456be5e2e-openshift-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:33:11.569256 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.569230 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73542c15-b3aa-449d-a2cd-a07456be5e2e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "73542c15-b3aa-449d-a2cd-a07456be5e2e" (UID: "73542c15-b3aa-449d-a2cd-a07456be5e2e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:33:11.569337 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.569241 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73542c15-b3aa-449d-a2cd-a07456be5e2e-kube-api-access-d7fmt" (OuterVolumeSpecName: "kube-api-access-d7fmt") pod "73542c15-b3aa-449d-a2cd-a07456be5e2e" (UID: "73542c15-b3aa-449d-a2cd-a07456be5e2e"). InnerVolumeSpecName "kube-api-access-d7fmt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:33:11.667921 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.667890 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7fmt\" (UniqueName: \"kubernetes.io/projected/73542c15-b3aa-449d-a2cd-a07456be5e2e-kube-api-access-d7fmt\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:33:11.667921 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.667915 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73542c15-b3aa-449d-a2cd-a07456be5e2e-maas-api-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:33:11.671022 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.670997 2571 generic.go:356] "Generic (PLEG): container finished" podID="73542c15-b3aa-449d-a2cd-a07456be5e2e" containerID="ddccdd2f653ca3333c71ee02e96b5f91b17c60e322ba2efc032422ca21c85924" exitCode=0 Jul 10 19:33:11.671151 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.671042 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" Jul 10 19:33:11.671151 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.671050 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" event={"ID":"73542c15-b3aa-449d-a2cd-a07456be5e2e","Type":"ContainerDied","Data":"ddccdd2f653ca3333c71ee02e96b5f91b17c60e322ba2efc032422ca21c85924"} Jul 10 19:33:11.671151 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.671090 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd" event={"ID":"73542c15-b3aa-449d-a2cd-a07456be5e2e","Type":"ContainerDied","Data":"7eded7c8f3632df239db14e5a370398edd82b62515f5aba08ad7d2e11b1ef63f"} Jul 10 19:33:11.671151 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.671124 2571 scope.go:117] "RemoveContainer" containerID="ddccdd2f653ca3333c71ee02e96b5f91b17c60e322ba2efc032422ca21c85924" Jul 10 19:33:11.683611 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.683592 2571 scope.go:117] "RemoveContainer" containerID="ddccdd2f653ca3333c71ee02e96b5f91b17c60e322ba2efc032422ca21c85924" Jul 10 19:33:11.683870 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:33:11.683847 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddccdd2f653ca3333c71ee02e96b5f91b17c60e322ba2efc032422ca21c85924\": container with ID starting with ddccdd2f653ca3333c71ee02e96b5f91b17c60e322ba2efc032422ca21c85924 not found: ID does not exist" containerID="ddccdd2f653ca3333c71ee02e96b5f91b17c60e322ba2efc032422ca21c85924" Jul 10 19:33:11.683921 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.683878 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddccdd2f653ca3333c71ee02e96b5f91b17c60e322ba2efc032422ca21c85924"} err="failed to get container status \"ddccdd2f653ca3333c71ee02e96b5f91b17c60e322ba2efc032422ca21c85924\": rpc error: code = NotFound desc = could not find container \"ddccdd2f653ca3333c71ee02e96b5f91b17c60e322ba2efc032422ca21c85924\": container with ID starting with ddccdd2f653ca3333c71ee02e96b5f91b17c60e322ba2efc032422ca21c85924 not found: ID does not exist" Jul 10 19:33:11.694291 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.694267 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd"] Jul 10 19:33:11.700662 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:11.700641 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-69b0c1-754b89cc55-72hfd"] Jul 10 19:33:13.018796 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:13.018764 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73542c15-b3aa-449d-a2cd-a07456be5e2e" path="/var/lib/kubelet/pods/73542c15-b3aa-449d-a2cd-a07456be5e2e/volumes" Jul 10 19:33:21.387331 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.387297 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47"] Jul 10 19:33:21.387930 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.387910 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73542c15-b3aa-449d-a2cd-a07456be5e2e" containerName="maas-api" Jul 10 19:33:21.387977 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.387935 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="73542c15-b3aa-449d-a2cd-a07456be5e2e" containerName="maas-api" Jul 10 19:33:21.388012 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.387988 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9bbf5ee-e2fb-410a-a0e7-ff792974ac60" containerName="cleanup" Jul 10 19:33:21.388012 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.387998 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9bbf5ee-e2fb-410a-a0e7-ff792974ac60" containerName="cleanup" Jul 10 19:33:21.388113 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.388086 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9bbf5ee-e2fb-410a-a0e7-ff792974ac60" containerName="cleanup" Jul 10 19:33:21.388153 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.388122 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="73542c15-b3aa-449d-a2cd-a07456be5e2e" containerName="maas-api" Jul 10 19:33:21.393115 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.393079 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" Jul 10 19:33:21.395798 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.395757 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-6669d3-serving-cert\"" Jul 10 19:33:21.395939 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.395848 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pj288\"" Jul 10 19:33:21.400803 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.400773 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47"] Jul 10 19:33:21.457337 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.457298 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c8n8\" (UniqueName: \"kubernetes.io/projected/5a9614a1-9651-4dd4-93e0-4ce777bb335d-kube-api-access-2c8n8\") pod \"maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47\" (UID: \"5a9614a1-9651-4dd4-93e0-4ce777bb335d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" Jul 10 19:33:21.457507 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.457437 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a9614a1-9651-4dd4-93e0-4ce777bb335d-openshift-service-ca\") pod \"maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47\" (UID: \"5a9614a1-9651-4dd4-93e0-4ce777bb335d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" Jul 10 19:33:21.457507 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.457489 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a9614a1-9651-4dd4-93e0-4ce777bb335d-maas-api-tls\") pod \"maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47\" (UID: \"5a9614a1-9651-4dd4-93e0-4ce777bb335d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" Jul 10 19:33:21.558392 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.558348 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c8n8\" (UniqueName: \"kubernetes.io/projected/5a9614a1-9651-4dd4-93e0-4ce777bb335d-kube-api-access-2c8n8\") pod \"maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47\" (UID: \"5a9614a1-9651-4dd4-93e0-4ce777bb335d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" Jul 10 19:33:21.558582 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.558430 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a9614a1-9651-4dd4-93e0-4ce777bb335d-openshift-service-ca\") pod \"maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47\" (UID: \"5a9614a1-9651-4dd4-93e0-4ce777bb335d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" Jul 10 19:33:21.558582 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.558459 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a9614a1-9651-4dd4-93e0-4ce777bb335d-maas-api-tls\") pod \"maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47\" (UID: \"5a9614a1-9651-4dd4-93e0-4ce777bb335d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" Jul 10 19:33:21.558702 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:33:21.558609 2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-serving-cert: secret "maas-api-e2e-api-a-6669d3-serving-cert" not found Jul 10 19:33:21.558702 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:33:21.558675 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a9614a1-9651-4dd4-93e0-4ce777bb335d-maas-api-tls podName:5a9614a1-9651-4dd4-93e0-4ce777bb335d nodeName:}" failed. No retries permitted until 2026-07-10 19:33:22.058654755 +0000 UTC m=+1983.594997165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5a9614a1-9651-4dd4-93e0-4ce777bb335d-maas-api-tls") pod "maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" (UID: "5a9614a1-9651-4dd4-93e0-4ce777bb335d") : secret "maas-api-e2e-api-a-6669d3-serving-cert" not found Jul 10 19:33:21.559204 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.559179 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a9614a1-9651-4dd4-93e0-4ce777bb335d-openshift-service-ca\") pod \"maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47\" (UID: \"5a9614a1-9651-4dd4-93e0-4ce777bb335d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" Jul 10 19:33:21.567709 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:21.567680 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c8n8\" (UniqueName: \"kubernetes.io/projected/5a9614a1-9651-4dd4-93e0-4ce777bb335d-kube-api-access-2c8n8\") pod \"maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47\" (UID: \"5a9614a1-9651-4dd4-93e0-4ce777bb335d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" Jul 10 19:33:22.063451 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:22.063414 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a9614a1-9651-4dd4-93e0-4ce777bb335d-maas-api-tls\") pod \"maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47\" (UID: \"5a9614a1-9651-4dd4-93e0-4ce777bb335d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" Jul 10 19:33:22.063647 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:33:22.063565 2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-serving-cert: secret "maas-api-e2e-api-a-6669d3-serving-cert" not found Jul 10 19:33:22.063749 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:33:22.063665 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a9614a1-9651-4dd4-93e0-4ce777bb335d-maas-api-tls podName:5a9614a1-9651-4dd4-93e0-4ce777bb335d nodeName:}" failed. No retries permitted until 2026-07-10 19:33:23.063649329 +0000 UTC m=+1984.599991739 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5a9614a1-9651-4dd4-93e0-4ce777bb335d-maas-api-tls") pod "maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" (UID: "5a9614a1-9651-4dd4-93e0-4ce777bb335d") : secret "maas-api-e2e-api-a-6669d3-serving-cert" not found Jul 10 19:33:23.075147 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:23.075077 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a9614a1-9651-4dd4-93e0-4ce777bb335d-maas-api-tls\") pod \"maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47\" (UID: \"5a9614a1-9651-4dd4-93e0-4ce777bb335d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" Jul 10 19:33:23.077504 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:23.077482 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a9614a1-9651-4dd4-93e0-4ce777bb335d-maas-api-tls\") pod \"maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47\" (UID: \"5a9614a1-9651-4dd4-93e0-4ce777bb335d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" Jul 10 19:33:23.207698 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:23.207658 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" Jul 10 19:33:23.351053 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:23.350965 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47"] Jul 10 19:33:23.352168 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:33:23.352134 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a9614a1_9651_4dd4_93e0_4ce777bb335d.slice/crio-028ee025583a94ade2dc91712560a5b343f9152607a770b1ff67110386a4ec65 WatchSource:0}: Error finding container 028ee025583a94ade2dc91712560a5b343f9152607a770b1ff67110386a4ec65: Status 404 returned error can't find the container with id 028ee025583a94ade2dc91712560a5b343f9152607a770b1ff67110386a4ec65 Jul 10 19:33:23.732184 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:23.732053 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" event={"ID":"5a9614a1-9651-4dd4-93e0-4ce777bb335d","Type":"ContainerStarted","Data":"028ee025583a94ade2dc91712560a5b343f9152607a770b1ff67110386a4ec65"} Jul 10 19:33:24.739775 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:24.739743 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" event={"ID":"5a9614a1-9651-4dd4-93e0-4ce777bb335d","Type":"ContainerStarted","Data":"4f74c3bbc010c8bfee311d1ba0c0550954f5f17348365844c28f971c66adec37"} Jul 10 19:33:24.740211 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:24.739883 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" Jul 10 19:33:24.758032 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:24.757987 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" podStartSLOduration=3.328025959 podStartE2EDuration="3.757970798s" totalImagesPullingTime="429.944839ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:33:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:33:24.756693534 +0000 UTC m=+1986.293035965" watchObservedRunningTime="2026-07-10 19:33:24.757970798 +0000 UTC m=+1986.294313230" Jul 10 19:33:28.615481 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:28.615444 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x"] Jul 10 19:33:28.619380 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:28.619357 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" Jul 10 19:33:28.622048 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:28.622025 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-094d6e-serving-cert\"" Jul 10 19:33:28.631622 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:28.631599 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x"] Jul 10 19:33:28.736123 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:28.736070 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4801d43-c6a1-478d-bbc4-7eeef9b381d4-maas-api-tls\") pod \"maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x\" (UID: \"b4801d43-c6a1-478d-bbc4-7eeef9b381d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" Jul 10 19:33:28.736288 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:28.736209 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4801d43-c6a1-478d-bbc4-7eeef9b381d4-openshift-service-ca\") pod \"maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x\" (UID: \"b4801d43-c6a1-478d-bbc4-7eeef9b381d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" Jul 10 19:33:28.736335 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:28.736313 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km5hp\" (UniqueName: \"kubernetes.io/projected/b4801d43-c6a1-478d-bbc4-7eeef9b381d4-kube-api-access-km5hp\") pod \"maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x\" (UID: \"b4801d43-c6a1-478d-bbc4-7eeef9b381d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" Jul 10 19:33:28.837714 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:28.837679 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4801d43-c6a1-478d-bbc4-7eeef9b381d4-openshift-service-ca\") pod \"maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x\" (UID: \"b4801d43-c6a1-478d-bbc4-7eeef9b381d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" Jul 10 19:33:28.837894 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:28.837789 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km5hp\" (UniqueName: \"kubernetes.io/projected/b4801d43-c6a1-478d-bbc4-7eeef9b381d4-kube-api-access-km5hp\") pod \"maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x\" (UID: \"b4801d43-c6a1-478d-bbc4-7eeef9b381d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" Jul 10 19:33:28.837894 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:28.837826 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4801d43-c6a1-478d-bbc4-7eeef9b381d4-maas-api-tls\") pod \"maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x\" (UID: \"b4801d43-c6a1-478d-bbc4-7eeef9b381d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" Jul 10 19:33:28.838486 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:28.838466 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4801d43-c6a1-478d-bbc4-7eeef9b381d4-openshift-service-ca\") pod \"maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x\" (UID: \"b4801d43-c6a1-478d-bbc4-7eeef9b381d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" Jul 10 19:33:28.840180 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:28.840159 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4801d43-c6a1-478d-bbc4-7eeef9b381d4-maas-api-tls\") pod \"maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x\" (UID: \"b4801d43-c6a1-478d-bbc4-7eeef9b381d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" Jul 10 19:33:28.846605 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:28.846576 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-km5hp\" (UniqueName: \"kubernetes.io/projected/b4801d43-c6a1-478d-bbc4-7eeef9b381d4-kube-api-access-km5hp\") pod \"maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x\" (UID: \"b4801d43-c6a1-478d-bbc4-7eeef9b381d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" Jul 10 19:33:28.930075 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:28.929984 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" Jul 10 19:33:29.075626 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:29.075597 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x"] Jul 10 19:33:29.078179 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:33:29.078149 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4801d43_c6a1_478d_bbc4_7eeef9b381d4.slice/crio-bf9e069e6fbdbb64441465ba68cddf092366375f9a688c078026a41d9ea8d273 WatchSource:0}: Error finding container bf9e069e6fbdbb64441465ba68cddf092366375f9a688c078026a41d9ea8d273: Status 404 returned error can't find the container with id bf9e069e6fbdbb64441465ba68cddf092366375f9a688c078026a41d9ea8d273 Jul 10 19:33:29.766280 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:29.766186 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" event={"ID":"b4801d43-c6a1-478d-bbc4-7eeef9b381d4","Type":"ContainerStarted","Data":"e7021c6290d97a2688b1f9c3f2f21559bab89ba538ba34bd5aebe94fcbbfab83"} Jul 10 19:33:29.766280 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:29.766243 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" event={"ID":"b4801d43-c6a1-478d-bbc4-7eeef9b381d4","Type":"ContainerStarted","Data":"bf9e069e6fbdbb64441465ba68cddf092366375f9a688c078026a41d9ea8d273"} Jul 10 19:33:29.766683 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:29.766286 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" Jul 10 19:33:29.785943 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:29.785892 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" podStartSLOduration=1.455181477 podStartE2EDuration="1.785876292s" totalImagesPullingTime="330.694815ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:33:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:33:29.784630424 +0000 UTC m=+1991.320972857" watchObservedRunningTime="2026-07-10 19:33:29.785876292 +0000 UTC m=+1991.322218769" Jul 10 19:33:30.750173 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:30.750143 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" Jul 10 19:33:35.778534 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:35.778500 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" Jul 10 19:33:41.864113 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:41.864073 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47"] Jul 10 19:33:41.864570 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:41.864336 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" podUID="5a9614a1-9651-4dd4-93e0-4ce777bb335d" containerName="maas-api" containerID="cri-o://4f74c3bbc010c8bfee311d1ba0c0550954f5f17348365844c28f971c66adec37" gracePeriod=30 Jul 10 19:33:42.119233 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.119174 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" Jul 10 19:33:42.165460 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.165433 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2c8n8\" (UniqueName: \"kubernetes.io/projected/5a9614a1-9651-4dd4-93e0-4ce777bb335d-kube-api-access-2c8n8\") pod \"5a9614a1-9651-4dd4-93e0-4ce777bb335d\" (UID: \"5a9614a1-9651-4dd4-93e0-4ce777bb335d\") " Jul 10 19:33:42.165460 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.165463 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a9614a1-9651-4dd4-93e0-4ce777bb335d-maas-api-tls\") pod \"5a9614a1-9651-4dd4-93e0-4ce777bb335d\" (UID: \"5a9614a1-9651-4dd4-93e0-4ce777bb335d\") " Jul 10 19:33:42.165672 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.165618 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a9614a1-9651-4dd4-93e0-4ce777bb335d-openshift-service-ca\") pod \"5a9614a1-9651-4dd4-93e0-4ce777bb335d\" (UID: \"5a9614a1-9651-4dd4-93e0-4ce777bb335d\") " Jul 10 19:33:42.166029 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.165993 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a9614a1-9651-4dd4-93e0-4ce777bb335d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5a9614a1-9651-4dd4-93e0-4ce777bb335d" (UID: "5a9614a1-9651-4dd4-93e0-4ce777bb335d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:33:42.167585 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.167556 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a9614a1-9651-4dd4-93e0-4ce777bb335d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5a9614a1-9651-4dd4-93e0-4ce777bb335d" (UID: "5a9614a1-9651-4dd4-93e0-4ce777bb335d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:33:42.167585 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.167569 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a9614a1-9651-4dd4-93e0-4ce777bb335d-kube-api-access-2c8n8" (OuterVolumeSpecName: "kube-api-access-2c8n8") pod "5a9614a1-9651-4dd4-93e0-4ce777bb335d" (UID: "5a9614a1-9651-4dd4-93e0-4ce777bb335d"). InnerVolumeSpecName "kube-api-access-2c8n8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:33:42.267131 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.267081 2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a9614a1-9651-4dd4-93e0-4ce777bb335d-openshift-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:33:42.267131 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.267124 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2c8n8\" (UniqueName: \"kubernetes.io/projected/5a9614a1-9651-4dd4-93e0-4ce777bb335d-kube-api-access-2c8n8\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:33:42.267131 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.267135 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a9614a1-9651-4dd4-93e0-4ce777bb335d-maas-api-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:33:42.832888 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.832853 2571 generic.go:356] "Generic (PLEG): container finished" podID="5a9614a1-9651-4dd4-93e0-4ce777bb335d" containerID="4f74c3bbc010c8bfee311d1ba0c0550954f5f17348365844c28f971c66adec37" exitCode=0 Jul 10 19:33:42.833051 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.832897 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" Jul 10 19:33:42.833051 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.832936 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" event={"ID":"5a9614a1-9651-4dd4-93e0-4ce777bb335d","Type":"ContainerDied","Data":"4f74c3bbc010c8bfee311d1ba0c0550954f5f17348365844c28f971c66adec37"} Jul 10 19:33:42.833051 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.832994 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47" event={"ID":"5a9614a1-9651-4dd4-93e0-4ce777bb335d","Type":"ContainerDied","Data":"028ee025583a94ade2dc91712560a5b343f9152607a770b1ff67110386a4ec65"} Jul 10 19:33:42.833051 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.833017 2571 scope.go:117] "RemoveContainer" containerID="4f74c3bbc010c8bfee311d1ba0c0550954f5f17348365844c28f971c66adec37" Jul 10 19:33:42.842509 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.842490 2571 scope.go:117] "RemoveContainer" containerID="4f74c3bbc010c8bfee311d1ba0c0550954f5f17348365844c28f971c66adec37" Jul 10 19:33:42.842758 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:33:42.842735 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f74c3bbc010c8bfee311d1ba0c0550954f5f17348365844c28f971c66adec37\": container with ID starting with 4f74c3bbc010c8bfee311d1ba0c0550954f5f17348365844c28f971c66adec37 not found: ID does not exist" containerID="4f74c3bbc010c8bfee311d1ba0c0550954f5f17348365844c28f971c66adec37" Jul 10 19:33:42.842805 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.842771 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f74c3bbc010c8bfee311d1ba0c0550954f5f17348365844c28f971c66adec37"} err="failed to get container status \"4f74c3bbc010c8bfee311d1ba0c0550954f5f17348365844c28f971c66adec37\": rpc error: code = NotFound desc = could not find container \"4f74c3bbc010c8bfee311d1ba0c0550954f5f17348365844c28f971c66adec37\": container with ID starting with 4f74c3bbc010c8bfee311d1ba0c0550954f5f17348365844c28f971c66adec37 not found: ID does not exist" Jul 10 19:33:42.854872 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.854850 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47"] Jul 10 19:33:42.858591 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:42.858570 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6669d3-7cd8c8d4fb-zck47"] Jul 10 19:33:43.018867 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:43.018833 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a9614a1-9651-4dd4-93e0-4ce777bb335d" path="/var/lib/kubelet/pods/5a9614a1-9651-4dd4-93e0-4ce777bb335d/volumes" Jul 10 19:33:49.212377 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.212339 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x"] Jul 10 19:33:49.212846 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.212684 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" podUID="b4801d43-c6a1-478d-bbc4-7eeef9b381d4" containerName="maas-api" containerID="cri-o://e7021c6290d97a2688b1f9c3f2f21559bab89ba538ba34bd5aebe94fcbbfab83" gracePeriod=30 Jul 10 19:33:49.487812 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.487786 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" Jul 10 19:33:49.533613 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.533572 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4801d43-c6a1-478d-bbc4-7eeef9b381d4-openshift-service-ca\") pod \"b4801d43-c6a1-478d-bbc4-7eeef9b381d4\" (UID: \"b4801d43-c6a1-478d-bbc4-7eeef9b381d4\") " Jul 10 19:33:49.533613 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.533618 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4801d43-c6a1-478d-bbc4-7eeef9b381d4-maas-api-tls\") pod \"b4801d43-c6a1-478d-bbc4-7eeef9b381d4\" (UID: \"b4801d43-c6a1-478d-bbc4-7eeef9b381d4\") " Jul 10 19:33:49.533858 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.533651 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-km5hp\" (UniqueName: \"kubernetes.io/projected/b4801d43-c6a1-478d-bbc4-7eeef9b381d4-kube-api-access-km5hp\") pod \"b4801d43-c6a1-478d-bbc4-7eeef9b381d4\" (UID: \"b4801d43-c6a1-478d-bbc4-7eeef9b381d4\") " Jul 10 19:33:49.534023 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.533998 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4801d43-c6a1-478d-bbc4-7eeef9b381d4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b4801d43-c6a1-478d-bbc4-7eeef9b381d4" (UID: "b4801d43-c6a1-478d-bbc4-7eeef9b381d4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:33:49.535832 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.535797 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4801d43-c6a1-478d-bbc4-7eeef9b381d4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b4801d43-c6a1-478d-bbc4-7eeef9b381d4" (UID: "b4801d43-c6a1-478d-bbc4-7eeef9b381d4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:33:49.535931 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.535833 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4801d43-c6a1-478d-bbc4-7eeef9b381d4-kube-api-access-km5hp" (OuterVolumeSpecName: "kube-api-access-km5hp") pod "b4801d43-c6a1-478d-bbc4-7eeef9b381d4" (UID: "b4801d43-c6a1-478d-bbc4-7eeef9b381d4"). InnerVolumeSpecName "kube-api-access-km5hp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:33:49.635408 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.635376 2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4801d43-c6a1-478d-bbc4-7eeef9b381d4-openshift-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:33:49.635408 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.635405 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4801d43-c6a1-478d-bbc4-7eeef9b381d4-maas-api-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:33:49.635408 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.635415 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-km5hp\" (UniqueName: \"kubernetes.io/projected/b4801d43-c6a1-478d-bbc4-7eeef9b381d4-kube-api-access-km5hp\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:33:49.868815 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.868773 2571 generic.go:356] "Generic (PLEG): container finished" podID="b4801d43-c6a1-478d-bbc4-7eeef9b381d4" containerID="e7021c6290d97a2688b1f9c3f2f21559bab89ba538ba34bd5aebe94fcbbfab83" exitCode=0 Jul 10 19:33:49.868815 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.868816 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" Jul 10 19:33:49.869032 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.868809 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" event={"ID":"b4801d43-c6a1-478d-bbc4-7eeef9b381d4","Type":"ContainerDied","Data":"e7021c6290d97a2688b1f9c3f2f21559bab89ba538ba34bd5aebe94fcbbfab83"} Jul 10 19:33:49.869032 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.868938 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x" event={"ID":"b4801d43-c6a1-478d-bbc4-7eeef9b381d4","Type":"ContainerDied","Data":"bf9e069e6fbdbb64441465ba68cddf092366375f9a688c078026a41d9ea8d273"} Jul 10 19:33:49.869032 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.868970 2571 scope.go:117] "RemoveContainer" containerID="e7021c6290d97a2688b1f9c3f2f21559bab89ba538ba34bd5aebe94fcbbfab83" Jul 10 19:33:49.878043 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.878024 2571 scope.go:117] "RemoveContainer" containerID="e7021c6290d97a2688b1f9c3f2f21559bab89ba538ba34bd5aebe94fcbbfab83" Jul 10 19:33:49.878332 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:33:49.878311 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7021c6290d97a2688b1f9c3f2f21559bab89ba538ba34bd5aebe94fcbbfab83\": container with ID starting with e7021c6290d97a2688b1f9c3f2f21559bab89ba538ba34bd5aebe94fcbbfab83 not found: ID does not exist" containerID="e7021c6290d97a2688b1f9c3f2f21559bab89ba538ba34bd5aebe94fcbbfab83" Jul 10 19:33:49.878381 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.878339 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7021c6290d97a2688b1f9c3f2f21559bab89ba538ba34bd5aebe94fcbbfab83"} err="failed to get container status \"e7021c6290d97a2688b1f9c3f2f21559bab89ba538ba34bd5aebe94fcbbfab83\": rpc error: code = NotFound desc = could not find container \"e7021c6290d97a2688b1f9c3f2f21559bab89ba538ba34bd5aebe94fcbbfab83\": container with ID starting with e7021c6290d97a2688b1f9c3f2f21559bab89ba538ba34bd5aebe94fcbbfab83 not found: ID does not exist" Jul 10 19:33:49.890655 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.890632 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x"] Jul 10 19:33:49.892362 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:49.892341 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-094d6e-dd8d7f947-6pb6x"] Jul 10 19:33:51.019457 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:51.019418 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4801d43-c6a1-478d-bbc4-7eeef9b381d4" path="/var/lib/kubelet/pods/b4801d43-c6a1-478d-bbc4-7eeef9b381d4/volumes" Jul 10 19:33:58.824317 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:58.824278 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6"] Jul 10 19:33:58.824887 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:58.824871 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4801d43-c6a1-478d-bbc4-7eeef9b381d4" containerName="maas-api" Jul 10 19:33:58.824932 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:58.824890 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4801d43-c6a1-478d-bbc4-7eeef9b381d4" containerName="maas-api" Jul 10 19:33:58.824932 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:58.824928 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a9614a1-9651-4dd4-93e0-4ce777bb335d" containerName="maas-api" Jul 10 19:33:58.824996 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:58.824937 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a9614a1-9651-4dd4-93e0-4ce777bb335d" containerName="maas-api" Jul 10 19:33:58.825038 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:58.825026 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4801d43-c6a1-478d-bbc4-7eeef9b381d4" containerName="maas-api" Jul 10 19:33:58.825070 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:58.825042 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a9614a1-9651-4dd4-93e0-4ce777bb335d" containerName="maas-api" Jul 10 19:33:58.828674 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:58.828652 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" Jul 10 19:33:58.831300 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:58.831275 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-b1f450-serving-cert\"" Jul 10 19:33:58.831409 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:58.831374 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pj288\"" Jul 10 19:33:58.838341 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:58.838315 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6"] Jul 10 19:33:58.928017 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:58.927990 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c6b0971-c844-4f86-a383-2efcc0b039a4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6\" (UID: \"9c6b0971-c844-4f86-a383-2efcc0b039a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" Jul 10 19:33:58.928195 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:58.928096 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfpnq\" (UniqueName: \"kubernetes.io/projected/9c6b0971-c844-4f86-a383-2efcc0b039a4-kube-api-access-xfpnq\") pod \"maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6\" (UID: \"9c6b0971-c844-4f86-a383-2efcc0b039a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" Jul 10 19:33:58.928195 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:58.928163 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c6b0971-c844-4f86-a383-2efcc0b039a4-maas-api-tls\") pod \"maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6\" (UID: \"9c6b0971-c844-4f86-a383-2efcc0b039a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" Jul 10 19:33:59.028891 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:59.028865 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c6b0971-c844-4f86-a383-2efcc0b039a4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6\" (UID: \"9c6b0971-c844-4f86-a383-2efcc0b039a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" Jul 10 19:33:59.029059 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:59.028916 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfpnq\" (UniqueName: \"kubernetes.io/projected/9c6b0971-c844-4f86-a383-2efcc0b039a4-kube-api-access-xfpnq\") pod \"maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6\" (UID: \"9c6b0971-c844-4f86-a383-2efcc0b039a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" Jul 10 19:33:59.029059 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:59.028968 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c6b0971-c844-4f86-a383-2efcc0b039a4-maas-api-tls\") pod \"maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6\" (UID: \"9c6b0971-c844-4f86-a383-2efcc0b039a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" Jul 10 19:33:59.029179 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:33:59.029086 2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-serving-cert: secret "maas-api-e2e-shared-a-b1f450-serving-cert" not found Jul 10 19:33:59.029179 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:33:59.029174 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c6b0971-c844-4f86-a383-2efcc0b039a4-maas-api-tls podName:9c6b0971-c844-4f86-a383-2efcc0b039a4 nodeName:}" failed. No retries permitted until 2026-07-10 19:33:59.529154818 +0000 UTC m=+2021.065497229 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9c6b0971-c844-4f86-a383-2efcc0b039a4-maas-api-tls") pod "maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" (UID: "9c6b0971-c844-4f86-a383-2efcc0b039a4") : secret "maas-api-e2e-shared-a-b1f450-serving-cert" not found Jul 10 19:33:59.029595 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:59.029568 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c6b0971-c844-4f86-a383-2efcc0b039a4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6\" (UID: \"9c6b0971-c844-4f86-a383-2efcc0b039a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" Jul 10 19:33:59.038149 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:59.038093 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfpnq\" (UniqueName: \"kubernetes.io/projected/9c6b0971-c844-4f86-a383-2efcc0b039a4-kube-api-access-xfpnq\") pod \"maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6\" (UID: \"9c6b0971-c844-4f86-a383-2efcc0b039a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" Jul 10 19:33:59.534490 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:59.534457 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c6b0971-c844-4f86-a383-2efcc0b039a4-maas-api-tls\") pod \"maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6\" (UID: \"9c6b0971-c844-4f86-a383-2efcc0b039a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" Jul 10 19:33:59.536933 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:59.536905 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c6b0971-c844-4f86-a383-2efcc0b039a4-maas-api-tls\") pod \"maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6\" (UID: \"9c6b0971-c844-4f86-a383-2efcc0b039a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" Jul 10 19:33:59.740073 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:59.740038 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" Jul 10 19:33:59.876050 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:59.876021 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6"] Jul 10 19:33:59.877993 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:33:59.877968 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c6b0971_c844_4f86_a383_2efcc0b039a4.slice/crio-115f116c93ef48d25c8c0dc16203ab4b14e07b13e10fbb7597477a2e9204aec8 WatchSource:0}: Error finding container 115f116c93ef48d25c8c0dc16203ab4b14e07b13e10fbb7597477a2e9204aec8: Status 404 returned error can't find the container with id 115f116c93ef48d25c8c0dc16203ab4b14e07b13e10fbb7597477a2e9204aec8 Jul 10 19:33:59.922018 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:33:59.921984 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" event={"ID":"9c6b0971-c844-4f86-a383-2efcc0b039a4","Type":"ContainerStarted","Data":"115f116c93ef48d25c8c0dc16203ab4b14e07b13e10fbb7597477a2e9204aec8"} Jul 10 19:34:00.927731 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:00.927693 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" event={"ID":"9c6b0971-c844-4f86-a383-2efcc0b039a4","Type":"ContainerStarted","Data":"494be8f2f8e1753250915a3a3535726b58ae9394dff9c2155d19e3d030725e6a"} Jul 10 19:34:00.928173 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:00.927819 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" Jul 10 19:34:00.947258 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:00.947204 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" podStartSLOduration=2.607455851 podStartE2EDuration="2.947189768s" totalImagesPullingTime="339.733917ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:33:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:34:00.94560421 +0000 UTC m=+2022.481946645" watchObservedRunningTime="2026-07-10 19:34:00.947189768 +0000 UTC m=+2022.483532199" Jul 10 19:34:06.845978 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:06.845943 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s"] Jul 10 19:34:06.849905 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:06.849886 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" Jul 10 19:34:06.852423 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:06.852400 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-b02417-serving-cert\"" Jul 10 19:34:06.861074 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:06.861045 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s"] Jul 10 19:34:06.937701 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:06.937674 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" Jul 10 19:34:07.003801 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:07.003770 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42f8326f-1a33-40eb-8fe2-5662b1dd840a-maas-api-tls\") pod \"maas-api-e2e-shared-b-b02417-76757cd578-m9t7s\" (UID: \"42f8326f-1a33-40eb-8fe2-5662b1dd840a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" Jul 10 19:34:07.003980 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:07.003913 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9f8v\" (UniqueName: \"kubernetes.io/projected/42f8326f-1a33-40eb-8fe2-5662b1dd840a-kube-api-access-v9f8v\") pod \"maas-api-e2e-shared-b-b02417-76757cd578-m9t7s\" (UID: \"42f8326f-1a33-40eb-8fe2-5662b1dd840a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" Jul 10 19:34:07.004091 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:07.004070 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42f8326f-1a33-40eb-8fe2-5662b1dd840a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b02417-76757cd578-m9t7s\" (UID: \"42f8326f-1a33-40eb-8fe2-5662b1dd840a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" Jul 10 19:34:07.105653 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:07.105563 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42f8326f-1a33-40eb-8fe2-5662b1dd840a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b02417-76757cd578-m9t7s\" (UID: \"42f8326f-1a33-40eb-8fe2-5662b1dd840a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" Jul 10 19:34:07.105817 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:07.105792 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42f8326f-1a33-40eb-8fe2-5662b1dd840a-maas-api-tls\") pod \"maas-api-e2e-shared-b-b02417-76757cd578-m9t7s\" (UID: \"42f8326f-1a33-40eb-8fe2-5662b1dd840a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" Jul 10 19:34:07.105889 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:07.105842 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9f8v\" (UniqueName: \"kubernetes.io/projected/42f8326f-1a33-40eb-8fe2-5662b1dd840a-kube-api-access-v9f8v\") pod \"maas-api-e2e-shared-b-b02417-76757cd578-m9t7s\" (UID: \"42f8326f-1a33-40eb-8fe2-5662b1dd840a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" Jul 10 19:34:07.106324 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:07.106301 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42f8326f-1a33-40eb-8fe2-5662b1dd840a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b02417-76757cd578-m9t7s\" (UID: \"42f8326f-1a33-40eb-8fe2-5662b1dd840a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" Jul 10 19:34:07.108132 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:07.108095 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42f8326f-1a33-40eb-8fe2-5662b1dd840a-maas-api-tls\") pod \"maas-api-e2e-shared-b-b02417-76757cd578-m9t7s\" (UID: \"42f8326f-1a33-40eb-8fe2-5662b1dd840a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" Jul 10 19:34:07.115530 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:07.115510 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9f8v\" (UniqueName: \"kubernetes.io/projected/42f8326f-1a33-40eb-8fe2-5662b1dd840a-kube-api-access-v9f8v\") pod \"maas-api-e2e-shared-b-b02417-76757cd578-m9t7s\" (UID: \"42f8326f-1a33-40eb-8fe2-5662b1dd840a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" Jul 10 19:34:07.163706 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:07.163677 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" Jul 10 19:34:07.308399 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:07.308372 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s"] Jul 10 19:34:07.310433 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:34:07.310405 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42f8326f_1a33_40eb_8fe2_5662b1dd840a.slice/crio-4d104b5f48a75a5052022895bc48713cf6bba48271cd9cc10757a88abcbbe943 WatchSource:0}: Error finding container 4d104b5f48a75a5052022895bc48713cf6bba48271cd9cc10757a88abcbbe943: Status 404 returned error can't find the container with id 4d104b5f48a75a5052022895bc48713cf6bba48271cd9cc10757a88abcbbe943 Jul 10 19:34:07.964224 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:07.964141 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" event={"ID":"42f8326f-1a33-40eb-8fe2-5662b1dd840a","Type":"ContainerStarted","Data":"4a39a087be750e9c0267fbdafc5d1af8d3bdc52efe7d3c61f5b97bad18758226"} Jul 10 19:34:07.964224 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:07.964197 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" event={"ID":"42f8326f-1a33-40eb-8fe2-5662b1dd840a","Type":"ContainerStarted","Data":"4d104b5f48a75a5052022895bc48713cf6bba48271cd9cc10757a88abcbbe943"} Jul 10 19:34:07.964655 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:07.964272 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" Jul 10 19:34:07.981701 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:07.981652 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" podStartSLOduration=1.601021966 podStartE2EDuration="1.981637192s" totalImagesPullingTime="380.615226ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:34:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:34:07.980409584 +0000 UTC m=+2029.516752016" watchObservedRunningTime="2026-07-10 19:34:07.981637192 +0000 UTC m=+2029.517979623" Jul 10 19:34:13.974090 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:34:13.974057 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" Jul 10 19:35:19.110090 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:35:19.109983 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:35:19.115951 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:35:19.115931 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:39:52.526733 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:52.526691 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7655-140463795809200"] Jul 10 19:39:52.530268 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:52.530250 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7655-140463795809200" Jul 10 19:39:52.533052 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:52.533032 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-q7hmp\"" Jul 10 19:39:52.537421 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:52.537115 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7655-140463795809200"] Jul 10 19:39:52.645575 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:52.645543 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5zs5\" (UniqueName: \"kubernetes.io/projected/9f5eaa20-e746-491a-9f3a-a94651abc3d4-kube-api-access-c5zs5\") pod \"test-curl-7655-140463795809200\" (UID: \"9f5eaa20-e746-491a-9f3a-a94651abc3d4\") " pod="openshift-ingress/test-curl-7655-140463795809200" Jul 10 19:39:52.746386 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:52.746351 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5zs5\" (UniqueName: \"kubernetes.io/projected/9f5eaa20-e746-491a-9f3a-a94651abc3d4-kube-api-access-c5zs5\") pod \"test-curl-7655-140463795809200\" (UID: \"9f5eaa20-e746-491a-9f3a-a94651abc3d4\") " pod="openshift-ingress/test-curl-7655-140463795809200" Jul 10 19:39:52.765197 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:52.765170 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5zs5\" (UniqueName: \"kubernetes.io/projected/9f5eaa20-e746-491a-9f3a-a94651abc3d4-kube-api-access-c5zs5\") pod \"test-curl-7655-140463795809200\" (UID: \"9f5eaa20-e746-491a-9f3a-a94651abc3d4\") " pod="openshift-ingress/test-curl-7655-140463795809200" Jul 10 19:39:52.840158 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:52.840132 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7655-140463795809200" Jul 10 19:39:52.971767 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:52.971742 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7655-140463795809200"] Jul 10 19:39:52.977194 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:39:52.977167 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f5eaa20_e746_491a_9f3a_a94651abc3d4.slice/crio-155f65a28e8781b6565375274f98bbc8b84099312bf13fc960aefd1648d7d981 WatchSource:0}: Error finding container 155f65a28e8781b6565375274f98bbc8b84099312bf13fc960aefd1648d7d981: Status 404 returned error can't find the container with id 155f65a28e8781b6565375274f98bbc8b84099312bf13fc960aefd1648d7d981 Jul 10 19:39:52.979005 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:52.978989 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 19:39:53.699470 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:53.699438 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7655-140463795809200" event={"ID":"9f5eaa20-e746-491a-9f3a-a94651abc3d4","Type":"ContainerStarted","Data":"155f65a28e8781b6565375274f98bbc8b84099312bf13fc960aefd1648d7d981"} Jul 10 19:39:54.708955 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:54.708921 2571 generic.go:356] "Generic (PLEG): container finished" podID="9f5eaa20-e746-491a-9f3a-a94651abc3d4" containerID="b61a9733ba26fb587d68ab838b3b3714c26f5c96bbc292e706cd5eff5f13494d" exitCode=0 Jul 10 19:39:54.709368 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:54.709002 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7655-140463795809200" event={"ID":"9f5eaa20-e746-491a-9f3a-a94651abc3d4","Type":"ContainerDied","Data":"b61a9733ba26fb587d68ab838b3b3714c26f5c96bbc292e706cd5eff5f13494d"} Jul 10 19:39:55.867520 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:55.867491 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7655-140463795809200" Jul 10 19:39:55.903164 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:55.903086 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7655-140463795809200_9f5eaa20-e746-491a-9f3a-a94651abc3d4/test-curl-7655-140463795809200/0.log" Jul 10 19:39:55.925781 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:55.925753 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7655-140463795809200"] Jul 10 19:39:55.931676 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:55.931656 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7655-140463795809200"] Jul 10 19:39:55.975131 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:55.975056 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5zs5\" (UniqueName: \"kubernetes.io/projected/9f5eaa20-e746-491a-9f3a-a94651abc3d4-kube-api-access-c5zs5\") pod \"9f5eaa20-e746-491a-9f3a-a94651abc3d4\" (UID: \"9f5eaa20-e746-491a-9f3a-a94651abc3d4\") " Jul 10 19:39:55.977044 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:55.977022 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f5eaa20-e746-491a-9f3a-a94651abc3d4-kube-api-access-c5zs5" (OuterVolumeSpecName: "kube-api-access-c5zs5") pod "9f5eaa20-e746-491a-9f3a-a94651abc3d4" (UID: "9f5eaa20-e746-491a-9f3a-a94651abc3d4"). InnerVolumeSpecName "kube-api-access-c5zs5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:39:56.076541 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:56.076514 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5zs5\" (UniqueName: \"kubernetes.io/projected/9f5eaa20-e746-491a-9f3a-a94651abc3d4-kube-api-access-c5zs5\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:39:56.720032 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:56.720002 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7655-140463795809200" Jul 10 19:39:56.720032 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:56.720014 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="155f65a28e8781b6565375274f98bbc8b84099312bf13fc960aefd1648d7d981" Jul 10 19:39:56.729877 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:39:56.729853 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7655-140463795809200\" is forbidden: User \"system:node:ip-10-0-131-229.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-229.ec2.internal' and this object" podUID="9f5eaa20-e746-491a-9f3a-a94651abc3d4" pod="openshift-ingress/test-curl-7655-140463795809200" Jul 10 19:39:57.018084 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:39:57.018015 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f5eaa20-e746-491a-9f3a-a94651abc3d4" path="/var/lib/kubelet/pods/9f5eaa20-e746-491a-9f3a-a94651abc3d4/volumes" Jul 10 19:40:02.726885 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:02.726848 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7655-140463795698688"] Jul 10 19:40:02.727347 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:02.727263 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f5eaa20-e746-491a-9f3a-a94651abc3d4" containerName="test-curl-7655-140463795809200" Jul 10 19:40:02.727347 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:02.727275 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f5eaa20-e746-491a-9f3a-a94651abc3d4" containerName="test-curl-7655-140463795809200" Jul 10 19:40:02.727347 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:02.727345 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f5eaa20-e746-491a-9f3a-a94651abc3d4" containerName="test-curl-7655-140463795809200" Jul 10 19:40:02.730476 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:02.730460 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7655-140463795698688" Jul 10 19:40:02.733294 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:02.733275 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-q7hmp\"" Jul 10 19:40:02.738341 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:02.738319 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7655-140463795698688"] Jul 10 19:40:02.840946 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:02.840908 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z94t\" (UniqueName: \"kubernetes.io/projected/45f480f3-c758-490e-a017-cec338f4be73-kube-api-access-5z94t\") pod \"test-curl-7655-140463795698688\" (UID: \"45f480f3-c758-490e-a017-cec338f4be73\") " pod="openshift-ingress/test-curl-7655-140463795698688" Jul 10 19:40:02.942295 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:02.942267 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z94t\" (UniqueName: \"kubernetes.io/projected/45f480f3-c758-490e-a017-cec338f4be73-kube-api-access-5z94t\") pod \"test-curl-7655-140463795698688\" (UID: \"45f480f3-c758-490e-a017-cec338f4be73\") " pod="openshift-ingress/test-curl-7655-140463795698688" Jul 10 19:40:02.950028 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:02.949995 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z94t\" (UniqueName: \"kubernetes.io/projected/45f480f3-c758-490e-a017-cec338f4be73-kube-api-access-5z94t\") pod \"test-curl-7655-140463795698688\" (UID: \"45f480f3-c758-490e-a017-cec338f4be73\") " pod="openshift-ingress/test-curl-7655-140463795698688" Jul 10 19:40:03.040155 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:03.040122 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7655-140463795698688" Jul 10 19:40:03.172085 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:03.172051 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7655-140463795698688"] Jul 10 19:40:03.175759 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:40:03.175727 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45f480f3_c758_490e_a017_cec338f4be73.slice/crio-58fb5a09890013b31e49a7a3e51cccfcd0580842106a09d9efba7f904d279337 WatchSource:0}: Error finding container 58fb5a09890013b31e49a7a3e51cccfcd0580842106a09d9efba7f904d279337: Status 404 returned error can't find the container with id 58fb5a09890013b31e49a7a3e51cccfcd0580842106a09d9efba7f904d279337 Jul 10 19:40:03.754939 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:03.754902 2571 generic.go:356] "Generic (PLEG): container finished" podID="45f480f3-c758-490e-a017-cec338f4be73" containerID="4600ba4dcee87146f81956c885c4c79ec14be3b6675d6a50760f4cebd20c61f2" exitCode=0 Jul 10 19:40:03.755304 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:03.754988 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7655-140463795698688" event={"ID":"45f480f3-c758-490e-a017-cec338f4be73","Type":"ContainerDied","Data":"4600ba4dcee87146f81956c885c4c79ec14be3b6675d6a50760f4cebd20c61f2"} Jul 10 19:40:03.755304 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:03.755031 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7655-140463795698688" event={"ID":"45f480f3-c758-490e-a017-cec338f4be73","Type":"ContainerStarted","Data":"58fb5a09890013b31e49a7a3e51cccfcd0580842106a09d9efba7f904d279337"} Jul 10 19:40:04.903040 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:04.903012 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7655-140463795698688" Jul 10 19:40:04.925512 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:04.925483 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7655-140463795698688_45f480f3-c758-490e-a017-cec338f4be73/test-curl-7655-140463795698688/0.log" Jul 10 19:40:04.949785 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:04.949758 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7655-140463795698688"] Jul 10 19:40:04.953547 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:04.953525 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7655-140463795698688"] Jul 10 19:40:05.060545 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:05.060513 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5z94t\" (UniqueName: \"kubernetes.io/projected/45f480f3-c758-490e-a017-cec338f4be73-kube-api-access-5z94t\") pod \"45f480f3-c758-490e-a017-cec338f4be73\" (UID: \"45f480f3-c758-490e-a017-cec338f4be73\") " Jul 10 19:40:05.062524 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:05.062502 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45f480f3-c758-490e-a017-cec338f4be73-kube-api-access-5z94t" (OuterVolumeSpecName: "kube-api-access-5z94t") pod "45f480f3-c758-490e-a017-cec338f4be73" (UID: "45f480f3-c758-490e-a017-cec338f4be73"). InnerVolumeSpecName "kube-api-access-5z94t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:40:05.162346 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:05.162310 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5z94t\" (UniqueName: \"kubernetes.io/projected/45f480f3-c758-490e-a017-cec338f4be73-kube-api-access-5z94t\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:40:05.766313 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:05.766283 2571 scope.go:117] "RemoveContainer" containerID="4600ba4dcee87146f81956c885c4c79ec14be3b6675d6a50760f4cebd20c61f2" Jul 10 19:40:05.766313 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:05.766304 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7655-140463795698688" Jul 10 19:40:07.018669 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:07.018626 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45f480f3-c758-490e-a017-cec338f4be73" path="/var/lib/kubelet/pods/45f480f3-c758-490e-a017-cec338f4be73/volumes" Jul 10 19:40:11.745736 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:11.745700 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6"] Jul 10 19:40:11.746287 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:11.745944 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" podUID="9c6b0971-c844-4f86-a383-2efcc0b039a4" containerName="maas-api" containerID="cri-o://494be8f2f8e1753250915a3a3535726b58ae9394dff9c2155d19e3d030725e6a" gracePeriod=30 Jul 10 19:40:11.988612 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:11.988585 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" Jul 10 19:40:12.131345 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:12.131306 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfpnq\" (UniqueName: \"kubernetes.io/projected/9c6b0971-c844-4f86-a383-2efcc0b039a4-kube-api-access-xfpnq\") pod \"9c6b0971-c844-4f86-a383-2efcc0b039a4\" (UID: \"9c6b0971-c844-4f86-a383-2efcc0b039a4\") " Jul 10 19:40:12.131538 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:12.131361 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c6b0971-c844-4f86-a383-2efcc0b039a4-maas-api-tls\") pod \"9c6b0971-c844-4f86-a383-2efcc0b039a4\" (UID: \"9c6b0971-c844-4f86-a383-2efcc0b039a4\") " Jul 10 19:40:12.131538 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:12.131466 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c6b0971-c844-4f86-a383-2efcc0b039a4-openshift-service-ca\") pod \"9c6b0971-c844-4f86-a383-2efcc0b039a4\" (UID: \"9c6b0971-c844-4f86-a383-2efcc0b039a4\") " Jul 10 19:40:12.131834 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:12.131804 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c6b0971-c844-4f86-a383-2efcc0b039a4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9c6b0971-c844-4f86-a383-2efcc0b039a4" (UID: "9c6b0971-c844-4f86-a383-2efcc0b039a4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:40:12.133452 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:12.133426 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c6b0971-c844-4f86-a383-2efcc0b039a4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9c6b0971-c844-4f86-a383-2efcc0b039a4" (UID: "9c6b0971-c844-4f86-a383-2efcc0b039a4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:40:12.133571 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:12.133469 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c6b0971-c844-4f86-a383-2efcc0b039a4-kube-api-access-xfpnq" (OuterVolumeSpecName: "kube-api-access-xfpnq") pod "9c6b0971-c844-4f86-a383-2efcc0b039a4" (UID: "9c6b0971-c844-4f86-a383-2efcc0b039a4"). InnerVolumeSpecName "kube-api-access-xfpnq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:40:12.232530 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:12.232492 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfpnq\" (UniqueName: \"kubernetes.io/projected/9c6b0971-c844-4f86-a383-2efcc0b039a4-kube-api-access-xfpnq\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:40:12.232530 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:12.232523 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c6b0971-c844-4f86-a383-2efcc0b039a4-maas-api-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:40:12.232530 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:12.232533 2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c6b0971-c844-4f86-a383-2efcc0b039a4-openshift-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:40:12.802332 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:12.802297 2571 generic.go:356] "Generic (PLEG): container finished" podID="9c6b0971-c844-4f86-a383-2efcc0b039a4" containerID="494be8f2f8e1753250915a3a3535726b58ae9394dff9c2155d19e3d030725e6a" exitCode=0 Jul 10 19:40:12.802734 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:12.802339 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" Jul 10 19:40:12.802734 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:12.802346 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" event={"ID":"9c6b0971-c844-4f86-a383-2efcc0b039a4","Type":"ContainerDied","Data":"494be8f2f8e1753250915a3a3535726b58ae9394dff9c2155d19e3d030725e6a"} Jul 10 19:40:12.802734 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:12.802387 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" event={"ID":"9c6b0971-c844-4f86-a383-2efcc0b039a4","Type":"ContainerDied","Data":"115f116c93ef48d25c8c0dc16203ab4b14e07b13e10fbb7597477a2e9204aec8"} Jul 10 19:40:12.802734 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:12.802410 2571 scope.go:117] "RemoveContainer" containerID="494be8f2f8e1753250915a3a3535726b58ae9394dff9c2155d19e3d030725e6a" Jul 10 19:40:12.812146 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:12.812125 2571 scope.go:117] "RemoveContainer" containerID="494be8f2f8e1753250915a3a3535726b58ae9394dff9c2155d19e3d030725e6a" Jul 10 19:40:12.812388 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:40:12.812371 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"494be8f2f8e1753250915a3a3535726b58ae9394dff9c2155d19e3d030725e6a\": container with ID starting with 494be8f2f8e1753250915a3a3535726b58ae9394dff9c2155d19e3d030725e6a not found: ID does not exist" containerID="494be8f2f8e1753250915a3a3535726b58ae9394dff9c2155d19e3d030725e6a" Jul 10 19:40:12.812446 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:12.812395 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"494be8f2f8e1753250915a3a3535726b58ae9394dff9c2155d19e3d030725e6a"} err="failed to get container status \"494be8f2f8e1753250915a3a3535726b58ae9394dff9c2155d19e3d030725e6a\": rpc error: code = NotFound desc = could not find container \"494be8f2f8e1753250915a3a3535726b58ae9394dff9c2155d19e3d030725e6a\": container with ID starting with 494be8f2f8e1753250915a3a3535726b58ae9394dff9c2155d19e3d030725e6a not found: ID does not exist" Jul 10 19:40:12.829154 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:12.827495 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6"] Jul 10 19:40:12.835737 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:12.835714 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6"] Jul 10 19:40:13.018189 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:13.018157 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c6b0971-c844-4f86-a383-2efcc0b039a4" path="/var/lib/kubelet/pods/9c6b0971-c844-4f86-a383-2efcc0b039a4/volumes" Jul 10 19:40:14.934053 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:14.934009 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b1f450-76cfb94845-grcn6" podUID="9c6b0971-c844-4f86-a383-2efcc0b039a4" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.52:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 10 19:40:19.115855 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:19.115714 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s"] Jul 10 19:40:19.148441 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:19.148410 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:40:19.152981 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:19.152962 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kgpdv_f217ac86-726b-4051-a7b4-aea1fe005f49/ovn-acl-logging/0.log" Jul 10 19:40:19.838843 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:19.838783 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" podUID="42f8326f-1a33-40eb-8fe2-5662b1dd840a" containerName="maas-api" containerID="cri-o://4a39a087be750e9c0267fbdafc5d1af8d3bdc52efe7d3c61f5b97bad18758226" gracePeriod=30 Jul 10 19:40:20.088220 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.088193 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" Jul 10 19:40:20.098524 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.098502 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42f8326f-1a33-40eb-8fe2-5662b1dd840a-openshift-service-ca\") pod \"42f8326f-1a33-40eb-8fe2-5662b1dd840a\" (UID: \"42f8326f-1a33-40eb-8fe2-5662b1dd840a\") " Jul 10 19:40:20.098677 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.098578 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9f8v\" (UniqueName: \"kubernetes.io/projected/42f8326f-1a33-40eb-8fe2-5662b1dd840a-kube-api-access-v9f8v\") pod \"42f8326f-1a33-40eb-8fe2-5662b1dd840a\" (UID: \"42f8326f-1a33-40eb-8fe2-5662b1dd840a\") " Jul 10 19:40:20.098677 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.098613 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42f8326f-1a33-40eb-8fe2-5662b1dd840a-maas-api-tls\") pod \"42f8326f-1a33-40eb-8fe2-5662b1dd840a\" (UID: \"42f8326f-1a33-40eb-8fe2-5662b1dd840a\") " Jul 10 19:40:20.098885 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.098861 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42f8326f-1a33-40eb-8fe2-5662b1dd840a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "42f8326f-1a33-40eb-8fe2-5662b1dd840a" (UID: "42f8326f-1a33-40eb-8fe2-5662b1dd840a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:40:20.100605 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.100578 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42f8326f-1a33-40eb-8fe2-5662b1dd840a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "42f8326f-1a33-40eb-8fe2-5662b1dd840a" (UID: "42f8326f-1a33-40eb-8fe2-5662b1dd840a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:40:20.100713 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.100654 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42f8326f-1a33-40eb-8fe2-5662b1dd840a-kube-api-access-v9f8v" (OuterVolumeSpecName: "kube-api-access-v9f8v") pod "42f8326f-1a33-40eb-8fe2-5662b1dd840a" (UID: "42f8326f-1a33-40eb-8fe2-5662b1dd840a"). InnerVolumeSpecName "kube-api-access-v9f8v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:40:20.200040 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.199997 2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42f8326f-1a33-40eb-8fe2-5662b1dd840a-openshift-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:40:20.200040 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.200038 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9f8v\" (UniqueName: \"kubernetes.io/projected/42f8326f-1a33-40eb-8fe2-5662b1dd840a-kube-api-access-v9f8v\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:40:20.200468 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.200052 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42f8326f-1a33-40eb-8fe2-5662b1dd840a-maas-api-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 19:40:20.845234 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.845197 2571 generic.go:356] "Generic (PLEG): container finished" podID="42f8326f-1a33-40eb-8fe2-5662b1dd840a" containerID="4a39a087be750e9c0267fbdafc5d1af8d3bdc52efe7d3c61f5b97bad18758226" exitCode=0 Jul 10 19:40:20.845423 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.845239 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" Jul 10 19:40:20.845423 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.845283 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" event={"ID":"42f8326f-1a33-40eb-8fe2-5662b1dd840a","Type":"ContainerDied","Data":"4a39a087be750e9c0267fbdafc5d1af8d3bdc52efe7d3c61f5b97bad18758226"} Jul 10 19:40:20.845423 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.845334 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s" event={"ID":"42f8326f-1a33-40eb-8fe2-5662b1dd840a","Type":"ContainerDied","Data":"4d104b5f48a75a5052022895bc48713cf6bba48271cd9cc10757a88abcbbe943"} Jul 10 19:40:20.845423 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.845354 2571 scope.go:117] "RemoveContainer" containerID="4a39a087be750e9c0267fbdafc5d1af8d3bdc52efe7d3c61f5b97bad18758226" Jul 10 19:40:20.854491 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.854472 2571 scope.go:117] "RemoveContainer" containerID="4a39a087be750e9c0267fbdafc5d1af8d3bdc52efe7d3c61f5b97bad18758226" Jul 10 19:40:20.854737 ip-10-0-131-229 kubenswrapper[2571]: E0710 19:40:20.854718 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a39a087be750e9c0267fbdafc5d1af8d3bdc52efe7d3c61f5b97bad18758226\": container with ID starting with 4a39a087be750e9c0267fbdafc5d1af8d3bdc52efe7d3c61f5b97bad18758226 not found: ID does not exist" containerID="4a39a087be750e9c0267fbdafc5d1af8d3bdc52efe7d3c61f5b97bad18758226" Jul 10 19:40:20.854783 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.854745 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a39a087be750e9c0267fbdafc5d1af8d3bdc52efe7d3c61f5b97bad18758226"} err="failed to get container status \"4a39a087be750e9c0267fbdafc5d1af8d3bdc52efe7d3c61f5b97bad18758226\": rpc error: code = NotFound desc = could not find container \"4a39a087be750e9c0267fbdafc5d1af8d3bdc52efe7d3c61f5b97bad18758226\": container with ID starting with 4a39a087be750e9c0267fbdafc5d1af8d3bdc52efe7d3c61f5b97bad18758226 not found: ID does not exist" Jul 10 19:40:20.868069 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.868044 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s"] Jul 10 19:40:20.872546 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:20.872525 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b02417-76757cd578-m9t7s"] Jul 10 19:40:21.018472 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:21.018444 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42f8326f-1a33-40eb-8fe2-5662b1dd840a" path="/var/lib/kubelet/pods/42f8326f-1a33-40eb-8fe2-5662b1dd840a/volumes" Jul 10 19:40:31.077221 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:31.077191 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-pmzsc_4bbbb813-070d-4dfa-ada4-0a72be142c3b/manager/0.log" Jul 10 19:40:31.202523 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:31.202486 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-b92cg_5112fd0d-a35d-4710-9fbc-9ba814d56e22/manager/0.log" Jul 10 19:40:31.703652 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:31.703620 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7668cd484b-58f6b_2c8f9311-d468-4ff4-a392-ca3db133731b/copy-manifests/0.log" Jul 10 19:40:31.722489 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:31.722461 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7668cd484b-58f6b_2c8f9311-d468-4ff4-a392-ca3db133731b/manager/0.log" Jul 10 19:40:32.461175 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:32.461143 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-wsffq_97650629-5712-4285-a7e5-3f7107cdb9bc/postgres/0.log" Jul 10 19:40:33.856400 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:33.856369 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-djx2g_4cb58ef8-03fe-499a-944e-af2e69a86c76/manager/0.log" Jul 10 19:40:34.223032 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:34.222950 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-btldc_0aaea817-77c7-4656-a317-da611d831777/registry-server/0.log" Jul 10 19:40:35.391232 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:35.391195 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-v2767_56f8901b-5cec-4f5a-8090-fbc9473159ac/istio-proxy/1.log" Jul 10 19:40:35.729160 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:35.729063 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68c78766fd-2z8xb_97aba93c-11ba-49b1-8d28-3e26078983b2/router/0.log" Jul 10 19:40:36.048480 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:36.048449 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-vr4lh_9a476776-c067-49f1-aeeb-e6619e36239d/main/0.log" Jul 10 19:40:36.160687 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:36.160665 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-sjq7r_57d5ded1-3ce5-4d9d-a518-264c8254bbcc/main/0.log" Jul 10 19:40:36.382655 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:36.382552 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblk6bk_eb08f413-7474-4391-b673-60a52eac1d7e/main/0.log" Jul 10 19:40:43.180622 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:43.180545 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4cdrm_a7f690e5-5f82-4573-bc6e-a7ed083f6fbc/global-pull-secret-syncer/0.log" Jul 10 19:40:43.420698 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:43.420671 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xm2hk_9fb6edd9-1cae-495c-9a2a-2da429edff2c/konnectivity-agent/0.log" Jul 10 19:40:43.475537 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:43.475459 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-229.ec2.internal_fe6cda58f405aaf9348408762755bc73/haproxy/0.log" Jul 10 19:40:49.150956 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:49.150926 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-djx2g_4cb58ef8-03fe-499a-944e-af2e69a86c76/manager/0.log" Jul 10 19:40:49.286309 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:49.286259 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-btldc_0aaea817-77c7-4656-a317-da611d831777/registry-server/0.log" Jul 10 19:40:51.137358 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.137312 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6798b0dd-81ab-42bd-9ed4-9811a97c0950/alertmanager/0.log" Jul 10 19:40:51.202830 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.202759 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6798b0dd-81ab-42bd-9ed4-9811a97c0950/config-reloader/0.log" Jul 10 19:40:51.229372 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.229352 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6798b0dd-81ab-42bd-9ed4-9811a97c0950/kube-rbac-proxy-web/0.log" Jul 10 19:40:51.254474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.254452 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6798b0dd-81ab-42bd-9ed4-9811a97c0950/kube-rbac-proxy/0.log" Jul 10 19:40:51.278746 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.278723 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6798b0dd-81ab-42bd-9ed4-9811a97c0950/kube-rbac-proxy-metric/0.log" Jul 10 19:40:51.303603 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.303582 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6798b0dd-81ab-42bd-9ed4-9811a97c0950/prom-label-proxy/0.log" Jul 10 19:40:51.331551 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.331527 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6798b0dd-81ab-42bd-9ed4-9811a97c0950/init-config-reloader/0.log" Jul 10 19:40:51.380474 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.380447 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-m7vls_3af4a62b-153c-4bc5-82f6-722a7f1e5259/cluster-monitoring-operator/0.log" Jul 10 19:40:51.413641 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.413617 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-6gl5d_3015d736-65f8-4b89-ad75-1f92c862b267/kube-state-metrics/0.log" Jul 10 19:40:51.438285 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.438265 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-6gl5d_3015d736-65f8-4b89-ad75-1f92c862b267/kube-rbac-proxy-main/0.log" Jul 10 19:40:51.462918 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.462860 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-6gl5d_3015d736-65f8-4b89-ad75-1f92c862b267/kube-rbac-proxy-self/0.log" Jul 10 19:40:51.492138 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.492120 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-78b7bbd787-cstjx_a64a56f1-0781-4eca-bd77-bb38a9af297f/metrics-server/0.log" Jul 10 19:40:51.560664 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.560642 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-79mdq_7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d/node-exporter/0.log" Jul 10 19:40:51.583230 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.583205 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-79mdq_7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d/kube-rbac-proxy/0.log" Jul 10 19:40:51.608437 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.608413 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-79mdq_7d6f77fb-4ad4-47c3-a5bd-636b192c0b4d/init-textfile/0.log" Jul 10 19:40:51.808658 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.808636 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-khf86_7f7d20e2-9db3-4e12-af8d-2b179b92e50b/kube-rbac-proxy-main/0.log" Jul 10 19:40:51.831308 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.831284 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-khf86_7f7d20e2-9db3-4e12-af8d-2b179b92e50b/kube-rbac-proxy-self/0.log" Jul 10 19:40:51.856833 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.856804 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-khf86_7f7d20e2-9db3-4e12-af8d-2b179b92e50b/openshift-state-metrics/0.log" Jul 10 19:40:51.911774 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.911746 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_164024dc-6cbc-4dce-b9f9-f3b4ea73ac56/prometheus/0.log" Jul 10 19:40:51.936529 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.936508 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_164024dc-6cbc-4dce-b9f9-f3b4ea73ac56/config-reloader/0.log" Jul 10 19:40:51.961592 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.961571 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_164024dc-6cbc-4dce-b9f9-f3b4ea73ac56/thanos-sidecar/0.log" Jul 10 19:40:51.988833 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:51.988818 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_164024dc-6cbc-4dce-b9f9-f3b4ea73ac56/kube-rbac-proxy-web/0.log" Jul 10 19:40:52.013586 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:52.013566 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_164024dc-6cbc-4dce-b9f9-f3b4ea73ac56/kube-rbac-proxy/0.log" Jul 10 19:40:52.038530 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:52.038499 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_164024dc-6cbc-4dce-b9f9-f3b4ea73ac56/kube-rbac-proxy-thanos/0.log" Jul 10 19:40:52.064144 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:52.064065 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_164024dc-6cbc-4dce-b9f9-f3b4ea73ac56/init-config-reloader/0.log" Jul 10 19:40:52.103160 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:52.103137 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-ng62p_489ea98d-c7b4-4517-9c64-251ead911737/prometheus-operator/0.log" Jul 10 19:40:52.124788 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:52.124758 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-ng62p_489ea98d-c7b4-4517-9c64-251ead911737/kube-rbac-proxy/0.log" Jul 10 19:40:52.161709 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:52.161690 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-sgckr_da2cf891-31e8-41b8-8a58-e8fce1192bdc/prometheus-operator-admission-webhook/0.log" Jul 10 19:40:52.194644 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:52.194618 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-64975d4fc5-mb6s6_379b8e10-3447-4632-af6c-8f392b1f23d5/telemeter-client/0.log" Jul 10 19:40:52.218062 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:52.218038 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-64975d4fc5-mb6s6_379b8e10-3447-4632-af6c-8f392b1f23d5/reload/0.log" Jul 10 19:40:52.246364 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:52.246344 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-64975d4fc5-mb6s6_379b8e10-3447-4632-af6c-8f392b1f23d5/kube-rbac-proxy/0.log" Jul 10 19:40:53.430225 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.430195 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d"] Jul 10 19:40:53.430683 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.430583 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45f480f3-c758-490e-a017-cec338f4be73" containerName="test-curl-7655-140463795698688" Jul 10 19:40:53.430683 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.430598 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="45f480f3-c758-490e-a017-cec338f4be73" containerName="test-curl-7655-140463795698688" Jul 10 19:40:53.430683 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.430630 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42f8326f-1a33-40eb-8fe2-5662b1dd840a" containerName="maas-api" Jul 10 19:40:53.430683 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.430636 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="42f8326f-1a33-40eb-8fe2-5662b1dd840a" containerName="maas-api" Jul 10 19:40:53.430683 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.430644 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c6b0971-c844-4f86-a383-2efcc0b039a4" containerName="maas-api" Jul 10 19:40:53.430683 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.430649 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c6b0971-c844-4f86-a383-2efcc0b039a4" containerName="maas-api" Jul 10 19:40:53.430971 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.430720 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="45f480f3-c758-490e-a017-cec338f4be73" containerName="test-curl-7655-140463795698688" Jul 10 19:40:53.430971 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.430731 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c6b0971-c844-4f86-a383-2efcc0b039a4" containerName="maas-api" Jul 10 19:40:53.430971 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.430746 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="42f8326f-1a33-40eb-8fe2-5662b1dd840a" containerName="maas-api" Jul 10 19:40:53.434201 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.434180 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:40:53.437240 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.437221 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z44r6\"/\"kube-root-ca.crt\"" Jul 10 19:40:53.438009 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.437990 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-z44r6\"/\"default-dockercfg-q7qdh\"" Jul 10 19:40:53.438082 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.437992 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z44r6\"/\"openshift-service-ca.crt\"" Jul 10 19:40:53.440801 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.440780 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d"] Jul 10 19:40:53.609289 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.609254 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86bfb447-6ee0-4cc0-8f59-1d3a95d47870-sys\") pod \"perf-node-gather-daemonset-mhx8d\" (UID: \"86bfb447-6ee0-4cc0-8f59-1d3a95d47870\") " pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:40:53.609460 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.609298 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/86bfb447-6ee0-4cc0-8f59-1d3a95d47870-lib-modules\") pod \"perf-node-gather-daemonset-mhx8d\" (UID: \"86bfb447-6ee0-4cc0-8f59-1d3a95d47870\") " pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:40:53.609460 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.609357 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/86bfb447-6ee0-4cc0-8f59-1d3a95d47870-proc\") pod \"perf-node-gather-daemonset-mhx8d\" (UID: \"86bfb447-6ee0-4cc0-8f59-1d3a95d47870\") " pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:40:53.609460 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.609401 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5nnh\" (UniqueName: \"kubernetes.io/projected/86bfb447-6ee0-4cc0-8f59-1d3a95d47870-kube-api-access-j5nnh\") pod \"perf-node-gather-daemonset-mhx8d\" (UID: \"86bfb447-6ee0-4cc0-8f59-1d3a95d47870\") " pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:40:53.609571 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.609540 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/86bfb447-6ee0-4cc0-8f59-1d3a95d47870-podres\") pod \"perf-node-gather-daemonset-mhx8d\" (UID: \"86bfb447-6ee0-4cc0-8f59-1d3a95d47870\") " pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:40:53.711180 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.711063 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/86bfb447-6ee0-4cc0-8f59-1d3a95d47870-podres\") pod \"perf-node-gather-daemonset-mhx8d\" (UID: \"86bfb447-6ee0-4cc0-8f59-1d3a95d47870\") " pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:40:53.711180 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.711160 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86bfb447-6ee0-4cc0-8f59-1d3a95d47870-sys\") pod \"perf-node-gather-daemonset-mhx8d\" (UID: \"86bfb447-6ee0-4cc0-8f59-1d3a95d47870\") " pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:40:53.711413 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.711185 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/86bfb447-6ee0-4cc0-8f59-1d3a95d47870-lib-modules\") pod \"perf-node-gather-daemonset-mhx8d\" (UID: \"86bfb447-6ee0-4cc0-8f59-1d3a95d47870\") " pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:40:53.711413 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.711206 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/86bfb447-6ee0-4cc0-8f59-1d3a95d47870-proc\") pod \"perf-node-gather-daemonset-mhx8d\" (UID: \"86bfb447-6ee0-4cc0-8f59-1d3a95d47870\") " pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:40:53.711413 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.711230 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5nnh\" (UniqueName: \"kubernetes.io/projected/86bfb447-6ee0-4cc0-8f59-1d3a95d47870-kube-api-access-j5nnh\") pod \"perf-node-gather-daemonset-mhx8d\" (UID: \"86bfb447-6ee0-4cc0-8f59-1d3a95d47870\") " pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:40:53.711413 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.711241 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/86bfb447-6ee0-4cc0-8f59-1d3a95d47870-podres\") pod \"perf-node-gather-daemonset-mhx8d\" (UID: \"86bfb447-6ee0-4cc0-8f59-1d3a95d47870\") " pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:40:53.711413 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.711284 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86bfb447-6ee0-4cc0-8f59-1d3a95d47870-sys\") pod \"perf-node-gather-daemonset-mhx8d\" (UID: \"86bfb447-6ee0-4cc0-8f59-1d3a95d47870\") " pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:40:53.711413 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.711326 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/86bfb447-6ee0-4cc0-8f59-1d3a95d47870-proc\") pod \"perf-node-gather-daemonset-mhx8d\" (UID: \"86bfb447-6ee0-4cc0-8f59-1d3a95d47870\") " pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:40:53.711413 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.711350 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/86bfb447-6ee0-4cc0-8f59-1d3a95d47870-lib-modules\") pod \"perf-node-gather-daemonset-mhx8d\" (UID: \"86bfb447-6ee0-4cc0-8f59-1d3a95d47870\") " pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:40:53.728298 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.728269 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5nnh\" (UniqueName: \"kubernetes.io/projected/86bfb447-6ee0-4cc0-8f59-1d3a95d47870-kube-api-access-j5nnh\") pod \"perf-node-gather-daemonset-mhx8d\" (UID: \"86bfb447-6ee0-4cc0-8f59-1d3a95d47870\") " pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:40:53.745271 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.745249 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:40:53.879664 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:53.879634 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d"] Jul 10 19:40:53.881628 ip-10-0-131-229 kubenswrapper[2571]: W0710 19:40:53.881585 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod86bfb447_6ee0_4cc0_8f59_1d3a95d47870.slice/crio-ce661e70191aad963ee3273dc7e77e4e0f6f6f98afdd4f3744112330f1afeafa WatchSource:0}: Error finding container ce661e70191aad963ee3273dc7e77e4e0f6f6f98afdd4f3744112330f1afeafa: Status 404 returned error can't find the container with id ce661e70191aad963ee3273dc7e77e4e0f6f6f98afdd4f3744112330f1afeafa Jul 10 19:40:54.014230 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:54.014134 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" event={"ID":"86bfb447-6ee0-4cc0-8f59-1d3a95d47870","Type":"ContainerStarted","Data":"c14a66ec783e62db6f87b3c47db759260a2d89c666d708172c10053871e73106"} Jul 10 19:40:54.014230 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:54.014212 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" event={"ID":"86bfb447-6ee0-4cc0-8f59-1d3a95d47870","Type":"ContainerStarted","Data":"ce661e70191aad963ee3273dc7e77e4e0f6f6f98afdd4f3744112330f1afeafa"} Jul 10 19:40:54.014397 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:54.014261 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:40:54.031424 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:54.031381 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" podStartSLOduration=1.031366965 podStartE2EDuration="1.031366965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:40:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:40:54.03120595 +0000 UTC m=+2435.567548382" watchObservedRunningTime="2026-07-10 19:40:54.031366965 +0000 UTC m=+2435.567709396" Jul 10 19:40:54.855710 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:54.855672 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-gb5kc_fd1dcfc5-bc32-448e-b38f-ce278c9992ff/download-server/0.log" Jul 10 19:40:56.555539 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:56.555513 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dptsw_50e43ef8-5c11-457b-b37c-ca6711ecd556/dns/0.log" Jul 10 19:40:56.577993 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:56.577971 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dptsw_50e43ef8-5c11-457b-b37c-ca6711ecd556/kube-rbac-proxy/0.log" Jul 10 19:40:56.882422 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:56.882349 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vrqsx_18f374ad-202d-43e8-b083-aabf526ff3d7/dns-node-resolver/0.log" Jul 10 19:40:57.539262 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:57.539233 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hzqbp_965c2d5d-ff87-46d2-9de2-7062e810316f/node-ca/0.log" Jul 10 19:40:58.785201 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:58.785167 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-v2767_56f8901b-5cec-4f5a-8090-fbc9473159ac/istio-proxy/0.log" Jul 10 19:40:58.842969 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:58.842939 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-v2767_56f8901b-5cec-4f5a-8090-fbc9473159ac/istio-proxy/1.log" Jul 10 19:40:58.928494 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:58.928466 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68c78766fd-2z8xb_97aba93c-11ba-49b1-8d28-3e26078983b2/router/0.log" Jul 10 19:40:59.666679 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:40:59.666648 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-47np5_a1bb26a4-858a-4c24-86ac-ef53647ded74/serve-healthcheck-canary/0.log" Jul 10 19:41:00.028678 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:41:00.028656 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-z44r6/perf-node-gather-daemonset-mhx8d" Jul 10 19:41:00.213688 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:41:00.213656 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-7h2bn_6ed7ba8a-9eab-45ed-944c-fd0c5d905551/insights-operator/0.log" Jul 10 19:41:00.214806 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:41:00.214780 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-7h2bn_6ed7ba8a-9eab-45ed-944c-fd0c5d905551/insights-operator/1.log" Jul 10 19:41:00.321013 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:41:00.320903 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dn2cb_92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10/kube-rbac-proxy/0.log" Jul 10 19:41:00.346030 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:41:00.346008 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dn2cb_92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10/exporter/0.log" Jul 10 19:41:00.368961 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:41:00.368936 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dn2cb_92b6ef1d-16c3-4c77-bfd3-882a3bd1cf10/extractor/0.log" Jul 10 19:41:02.931884 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:41:02.931855 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-pmzsc_4bbbb813-070d-4dfa-ada4-0a72be142c3b/manager/0.log" Jul 10 19:41:03.171137 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:41:03.171108 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-b92cg_5112fd0d-a35d-4710-9fbc-9ba814d56e22/manager/0.log" Jul 10 19:41:03.487553 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:41:03.487517 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7668cd484b-58f6b_2c8f9311-d468-4ff4-a392-ca3db133731b/manager/0.log" Jul 10 19:41:03.506580 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:41:03.506557 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7668cd484b-58f6b_2c8f9311-d468-4ff4-a392-ca3db133731b/copy-manifests/0.log" Jul 10 19:41:04.205513 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:41:04.205476 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-v2767_56f8901b-5cec-4f5a-8090-fbc9473159ac/istio-proxy/0.log" Jul 10 19:41:04.264022 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:41:04.263993 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-v2767_56f8901b-5cec-4f5a-8090-fbc9473159ac/istio-proxy/1.log" Jul 10 19:41:04.322921 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:41:04.322891 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68c78766fd-2z8xb_97aba93c-11ba-49b1-8d28-3e26078983b2/router/0.log" Jul 10 19:41:04.918505 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:41:04.918473 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6d9b94cf9d-cv2fz_44a59330-6b55-4e66-b169-71101482d022/manager/0.log" Jul 10 19:41:05.770387 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:41:05.770360 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-v2767_56f8901b-5cec-4f5a-8090-fbc9473159ac/istio-proxy/0.log" Jul 10 19:41:05.829589 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:41:05.829556 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-v2767_56f8901b-5cec-4f5a-8090-fbc9473159ac/istio-proxy/1.log" Jul 10 19:41:05.889437 ip-10-0-131-229 kubenswrapper[2571]: I0710 19:41:05.889404 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68c78766fd-2z8xb_97aba93c-11ba-49b1-8d28-3e26078983b2/router/0.log"