Jul 13 19:21:44.775171 ip-10-0-132-20 systemd[1]: Starting Kubernetes Kubelet... Jul 13 19:21:45.179759 ip-10-0-132-20 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 19:21:45.179759 ip-10-0-132-20 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 19:21:45.179759 ip-10-0-132-20 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 19:21:45.179759 ip-10-0-132-20 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 19:21:45.179759 ip-10-0-132-20 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 19:21:45.179759 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.149126 2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 19:21:45.179759 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153091 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 19:21:45.179759 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153115 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 19:21:45.179759 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153120 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 19:21:45.179759 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153123 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 19:21:45.179759 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153126 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 19:21:45.179759 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153130 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 19:21:45.179759 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153132 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 19:21:45.179759 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153135 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153138 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153142 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153146 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153149 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153152 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153154 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153157 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153160 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153163 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153166 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153170 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153175 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153178 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153187 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153191 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153194 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153197 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153200 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 19:21:45.181149 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153203 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153206 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153209 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153212 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153215 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153218 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153221 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153225 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153228 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153231 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153234 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153237 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153240 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153243 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153245 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153248 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153251 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153254 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153258 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153261 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 19:21:45.182080 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153264 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153267 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153270 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153273 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153276 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153278 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153281 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153284 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153287 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153290 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153293 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153297 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153300 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153305 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153310 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153315 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153319 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153323 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153326 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153330 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 19:21:45.183048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153333 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153336 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153339 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153343 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153346 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153350 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153353 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153357 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153360 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153363 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153366 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153368 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153371 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153374 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153377 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153380 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153384 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153386 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153389 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153393 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 19:21:45.183940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153397 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153401 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153404 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153407 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153410 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153414 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153417 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153419 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153423 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153426 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153429 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153431 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.153436 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154369 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154377 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154381 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154385 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154388 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154393 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154396 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 19:21:45.184835 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154399 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154402 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154405 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154409 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154412 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154415 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154417 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154421 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154424 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154427 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154429 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154433 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154436 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154439 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154442 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154444 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154448 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154451 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154454 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154457 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 19:21:45.185739 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154462 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154467 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154471 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154474 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154478 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154482 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154485 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154489 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154492 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154495 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154498 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154502 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154505 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154508 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154511 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154514 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154517 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154520 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154522 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154525 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 19:21:45.186603 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154528 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154531 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154534 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154537 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154540 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154542 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154547 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154550 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154553 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154556 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154559 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154562 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154565 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154568 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154572 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154575 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154578 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154581 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154583 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 19:21:45.187492 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154587 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154590 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154593 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154596 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154600 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154603 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154607 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154610 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154613 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154616 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154619 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154621 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154624 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154634 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154637 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154640 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154643 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154646 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154649 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154652 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 19:21:45.188389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154654 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154657 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154660 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154663 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154668 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154671 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154674 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154678 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154681 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154683 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154686 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154689 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.154692 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155198 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155204 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155207 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155211 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155214 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155217 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155220 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 19:21:45.189705 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155223 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155227 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155230 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155240 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155243 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155248 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155250 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155253 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155256 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155259 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155262 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155265 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155269 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155271 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155275 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155277 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155281 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155284 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155286 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155289 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 19:21:45.191039 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155292 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155295 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155298 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155300 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155303 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155305 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155308 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155311 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155315 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155317 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155321 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155323 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155326 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155329 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155332 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155336 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155339 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155345 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155347 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155350 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 19:21:45.192043 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155353 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155356 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155359 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155361 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155364 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155367 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155370 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155373 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155376 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155379 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155381 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155384 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155387 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155390 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155393 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155395 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155398 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155401 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155404 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 19:21:45.192984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155407 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155409 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155412 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155414 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155418 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155423 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155427 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155430 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155435 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155438 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155443 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155446 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155449 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155452 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155456 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155458 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155461 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155464 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155467 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 19:21:45.193886 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155470 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155473 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155475 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155478 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155480 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155483 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155487 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155491 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155494 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155497 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155501 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155505 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155508 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.155511 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155623 2556 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155635 2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155645 2556 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155651 2556 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155659 2556 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155662 2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 19:21:45.194895 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155667 2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155674 2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155678 2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155683 2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155687 2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155691 2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155694 2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155698 2556 flags.go:64] FLAG: --cgroup-root="" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155701 2556 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155704 2556 flags.go:64] FLAG: --client-ca-file="" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155707 2556 flags.go:64] FLAG: --cloud-provider="external" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155711 2556 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155716 2556 flags.go:64] FLAG: --cluster-domain="" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155720 2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155723 2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155727 2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155731 2556 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155735 2556 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155738 2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155742 2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155746 2556 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155749 2556 flags.go:64] FLAG: --contention-profiling="false" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155753 2556 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155756 2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 19:21:45.195784 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155760 2556 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155763 2556 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155768 2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155771 2556 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155774 2556 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155777 2556 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155781 2556 flags.go:64] FLAG: --enable-server="true" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155785 2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155790 2556 flags.go:64] FLAG: --event-burst="100" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155795 2556 flags.go:64] FLAG: --event-qps="50" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155798 2556 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155804 2556 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155808 2556 flags.go:64] FLAG: --eviction-hard="" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155812 2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155816 2556 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155820 2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155823 2556 flags.go:64] FLAG: --eviction-soft="" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155826 2556 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155829 2556 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155833 2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155836 2556 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155840 2556 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155843 2556 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155864 2556 flags.go:64] FLAG: --feature-gates="" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155870 2556 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 19:21:45.196733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155875 2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155878 2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155882 2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155885 2556 flags.go:64] FLAG: --healthz-port="10248" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155888 2556 flags.go:64] FLAG: --help="false" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155891 2556 flags.go:64] FLAG: --hostname-override="ip-10-0-132-20.ec2.internal" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155895 2556 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155899 2556 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155902 2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155905 2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155910 2556 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155913 2556 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155917 2556 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155920 2556 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155924 2556 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155929 2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155934 2556 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155937 2556 flags.go:64] FLAG: --kube-reserved="" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155942 2556 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155945 2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155949 2556 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155952 2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155955 2556 flags.go:64] FLAG: --lock-file="" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155958 2556 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 19:21:45.197639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155961 2556 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155965 2556 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155971 2556 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155975 2556 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155978 2556 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155981 2556 flags.go:64] FLAG: --logging-format="text" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155984 2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155988 2556 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155992 2556 flags.go:64] FLAG: --manifest-url="" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.155995 2556 flags.go:64] FLAG: --manifest-url-header="" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156000 2556 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156003 2556 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156008 2556 flags.go:64] FLAG: --max-pods="110" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156012 2556 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156015 2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156018 2556 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156022 2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156025 2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156028 2556 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156031 2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156040 2556 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156045 2556 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156048 2556 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 19:21:45.203672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156052 2556 flags.go:64] FLAG: --pod-cidr="" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156058 2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156064 2556 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156068 2556 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156073 2556 flags.go:64] FLAG: --pods-per-core="0" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156077 2556 flags.go:64] FLAG: --port="10250" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156080 2556 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156083 2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0075abd351d6a62dd" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156087 2556 flags.go:64] FLAG: --qos-reserved="" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156090 2556 flags.go:64] FLAG: --read-only-port="10255" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156093 2556 flags.go:64] FLAG: --register-node="true" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156097 2556 flags.go:64] FLAG: --register-with-taints="" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156102 2556 flags.go:64] FLAG: --registry-burst="10" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156105 2556 flags.go:64] FLAG: --registry-qps="5" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156108 2556 flags.go:64] FLAG: --reserved-cpus="" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156112 2556 flags.go:64] FLAG: --reserved-memory="" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156116 2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156119 2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156122 2556 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156126 2556 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156129 2556 flags.go:64] FLAG: --runonce="false" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156132 2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156136 2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156139 2556 flags.go:64] FLAG: --seccomp-default="false" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156142 2556 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 19:21:45.312199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156145 2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156148 2556 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156152 2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156155 2556 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156158 2556 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156162 2556 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156165 2556 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156168 2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156173 2556 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156177 2556 flags.go:64] FLAG: --system-cgroups="" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156180 2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156188 2556 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156192 2556 flags.go:64] FLAG: --tls-cert-file="" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156195 2556 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156201 2556 flags.go:64] FLAG: --tls-min-version="" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156204 2556 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156208 2556 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156211 2556 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156214 2556 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156217 2556 flags.go:64] FLAG: --v="2" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156222 2556 flags.go:64] FLAG: --version="false" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156226 2556 flags.go:64] FLAG: --vmodule="" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156231 2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156234 2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156375 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 19:21:45.324873 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156380 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156384 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156388 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156391 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156393 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156396 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156399 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156402 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156405 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156408 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156411 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156413 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156416 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156419 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156422 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156426 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156429 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156433 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156437 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156441 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 19:21:45.325737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156444 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156446 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156449 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156452 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156456 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156459 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156461 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156464 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156467 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156470 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156473 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156476 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156478 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156481 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156484 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156487 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156491 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156495 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156498 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 19:21:45.326627 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156501 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156504 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156508 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156511 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156514 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156517 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156520 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156524 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156529 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156532 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156535 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156544 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156547 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156550 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156553 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156556 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156559 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156562 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156565 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 19:21:45.327497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156568 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156571 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156574 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156577 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156580 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156582 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156585 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156588 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156591 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156595 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156598 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156602 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156605 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156608 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156611 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156613 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156616 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156619 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156623 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 19:21:45.328378 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156628 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156631 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156636 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156639 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156641 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156646 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156650 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156653 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156656 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156659 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156662 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156664 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156667 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156670 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156673 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156676 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156679 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156682 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156685 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156688 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 19:21:45.329298 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.156690 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 19:21:45.330182 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.156697 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 19:21:45.330182 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.165223 2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 19:21:45.330182 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.165254 2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 19:21:45.330182 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.166049 2556 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 19:21:45.330182 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.166067 2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 19:21:45.330182 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166163 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 19:21:45.330182 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166173 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 19:21:45.330182 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166180 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 19:21:45.330182 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166184 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 19:21:45.330182 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166189 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 19:21:45.330182 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166194 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 19:21:45.330182 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166199 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 19:21:45.330182 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166203 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 19:21:45.330182 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166207 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 19:21:45.330182 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166212 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166218 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166224 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166229 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166236 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166242 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166249 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166257 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166261 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166267 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166271 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166275 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166281 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166285 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166290 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166295 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166299 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166303 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166307 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 19:21:45.331525 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166311 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166317 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166321 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166326 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166330 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166335 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166340 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166344 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166348 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166352 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166357 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166361 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166366 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166370 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166374 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166378 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166382 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166388 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166393 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166398 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 19:21:45.332403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166403 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166407 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166411 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166416 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166421 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166425 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166429 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166434 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166439 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166443 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166448 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166452 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166457 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166461 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166467 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166472 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166476 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166481 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166485 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166489 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 19:21:45.333285 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166493 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166497 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166502 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166506 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166510 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166514 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166519 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166523 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166527 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166531 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166536 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166541 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166545 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166553 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166560 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166565 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166569 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166574 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166578 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 19:21:45.334142 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166583 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 19:21:45.335448 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166587 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 19:21:45.335448 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166591 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 19:21:45.335448 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166595 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 19:21:45.335448 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166600 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 19:21:45.335448 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166604 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 19:21:45.335448 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166608 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 19:21:45.335448 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166613 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 13 19:21:45.335448 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166618 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 19:21:45.335448 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166623 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 19:21:45.335448 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166628 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 19:21:45.335448 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166632 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 19:21:45.335448 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.166641 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 19:21:45.335448 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166867 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 19:21:45.335448 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166877 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 19:21:45.335448 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166883 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166887 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166892 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166897 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166902 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166906 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166910 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166915 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166920 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166925 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166929 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166937 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166941 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166945 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166949 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166953 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166957 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166962 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166966 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166971 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 19:21:45.336347 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166975 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166979 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166983 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166987 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166992 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.166995 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167001 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167006 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167010 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167014 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167020 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167024 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167028 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167032 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167036 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167041 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167044 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167049 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167054 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167058 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167063 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 19:21:45.337240 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167067 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 19:21:45.338585 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167074 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 19:21:45.338585 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167085 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 19:21:45.338585 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167090 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 19:21:45.338585 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167094 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 19:21:45.338585 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167098 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 19:21:45.338585 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167103 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 19:21:45.338585 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167110 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 19:21:45.338585 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167116 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 19:21:45.338585 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167121 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 19:21:45.338585 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167126 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 19:21:45.338585 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167130 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 19:21:45.338585 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167134 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 19:21:45.338585 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167141 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 19:21:45.338585 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167147 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 19:21:45.338585 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167151 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 19:21:45.338585 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167156 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 19:21:45.338585 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167161 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 19:21:45.338585 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167166 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167171 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167175 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167183 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167187 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167192 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167196 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167200 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167204 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167208 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167212 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167217 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167221 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167225 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167230 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167234 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167239 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167243 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167248 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167253 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 19:21:45.339451 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167257 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 19:21:45.340340 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167261 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 19:21:45.340340 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167265 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 19:21:45.340340 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167270 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 19:21:45.340340 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167274 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 19:21:45.340340 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167278 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 19:21:45.340340 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167282 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 19:21:45.340340 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167287 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 19:21:45.340340 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167291 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 19:21:45.340340 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167295 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 19:21:45.340340 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167299 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 19:21:45.340340 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167303 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 19:21:45.340340 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167308 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 19:21:45.340340 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167320 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 19:21:45.340340 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167326 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 19:21:45.340340 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167333 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 19:21:45.340340 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167339 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 19:21:45.340340 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:45.167344 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 19:21:45.341237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.167352 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 19:21:45.341237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.167564 2556 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 19:21:45.341237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.171550 2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 19:21:45.341237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.173044 2556 server.go:1019] "Starting client certificate rotation" Jul 13 19:21:45.341237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.173169 2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 19:21:45.341237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.173963 2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 19:21:45.341237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.175814 2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 19:21:45.341237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.177229 2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 19:21:45.341237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.191085 2556 log.go:25] "Validated CRI v1 runtime API" Jul 13 19:21:45.341237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.193768 2556 log.go:25] "Validated CRI v1 image API" Jul 13 19:21:45.341237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.194859 2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 19:21:45.341237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.199346 2556 fs.go:135] Filesystem UUIDs: map[71d29f2d-07d2-4f1b-b4d4-01f8b88e2889:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 bf0c45eb-32c4-4768-98a4-1881ccace7bd:/dev/nvme0n1p4] Jul 13 19:21:45.341654 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.199375 2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 19:21:45.342107 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.205300 2556 manager.go:216] Machine: {Timestamp:2026-07-13 19:21:45.204728977 +0000 UTC m=+0.318636679 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102390 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a5bc117fbbd1076921f40bb2b2d81 SystemUUID:ec2a5bc1-17fb-bd10-7692-1f40bb2b2d81 BootID:e3ca43cb-0269-41ca-b5b6-093b7135fa91 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:df:69:56:ed:19 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:df:69:56:ed:19 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:2e:b7:14:dc:d3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 19:21:45.342107 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.205421 2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 19:21:45.342107 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.205529 2556 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 19:21:45.342107 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.208566 2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 19:21:45.342107 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.208564 2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 19:21:45.342107 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.208606 2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-20.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 19:21:45.342107 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.210070 2556 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 19:21:45.453114 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.210088 2556 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 19:21:45.453114 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.210100 2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 19:21:45.453114 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.210765 2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 19:21:45.453114 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.233665 2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dg4wd" Jul 13 19:21:45.453114 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.244323 2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dg4wd" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.743638 2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.747826 2556 state_mem.go:36] "Initialized new in-memory state store" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.748114 2556 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.748163 2556 kubelet.go:486] "Attempting to sync node with API server" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.748188 2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.748209 2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.748224 2556 kubelet.go:398] "Adding apiserver pod source" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.748240 2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.748824 2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.748889 2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.753946 2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.755448 2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.755479 2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.756161 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.756179 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.756186 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.756192 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.756201 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.756207 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.756217 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.756223 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.756231 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.756237 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.756247 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.756253 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.756279 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.756284 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.759908 2556 server.go:1268] "Started kubelet" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.759979 2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.760001 2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.760085 2556 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.761341 2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.762748 2556 server.go:310] "Adding debug handlers to kubelet server" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.766415 2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.766454 2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:45.766709 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-20.ec2.internal\" not found" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.767087 2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.767092 2556 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.767115 2556 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.767260 2556 reconstruct.go:97] "Volume reconstruction finished" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.767268 2556 reconciler.go:29] "Reconciler: start to sync state" Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.767692 2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.767715 2556 factory.go:55] Registering systemd factory Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.767726 2556 factory.go:223] Registration of the systemd container factory successfully Jul 13 19:21:45.869775 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:45.767698 2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 19:21:45.760969 ip-10-0-132-20 systemd[1]: Started Kubernetes Kubelet. Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.767989 2556 factory.go:153] Registering CRI-O factory Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.768000 2556 factory.go:223] Registration of the crio container factory successfully Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.768027 2556 factory.go:103] Registering Raw factory Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.768042 2556 manager.go:1196] Started watching for new ooms in manager Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.768782 2556 manager.go:349] Starting recovery of all containers Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.772169 2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.772915 2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.782617 2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.783122 2556 manager.go:354] Recovery completed Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.788688 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.790905 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-20.ec2.internal" event="NodeHasSufficientMemory" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.790942 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-20.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.790957 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-20.ec2.internal" event="NodeHasSufficientPID" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.791511 2556 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.791520 2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.791526 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.791541 2556 state_mem.go:36] "Initialized new in-memory state store" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.792091 2556 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-132-20.ec2.internal" not found Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:45.792126 2556 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-20.ec2.internal\" not found" node="ip-10-0-132-20.ec2.internal" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.792319 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-20.ec2.internal" event="NodeHasSufficientMemory" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.792346 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-20.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.792358 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-20.ec2.internal" event="NodeHasSufficientPID" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.793033 2556 policy_none.go:49] "None policy: Start" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.793050 2556 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.793061 2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.794066 2556 policy_none.go:47] "Start" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.813041 2556 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-132-20.ec2.internal" not found Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.839800 2556 manager.go:337] "Starting Device Plugin manager" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:45.839842 2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.839871 2556 server.go:87] "Starting device plugin registration server" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.840196 2556 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.840209 2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.840303 2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.840395 2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.840405 2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:45.840934 2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 19:21:45.871678 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:45.840999 2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-20.ec2.internal\" not found" Jul 13 19:21:46.004593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.878134 2556 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-132-20.ec2.internal" not found Jul 13 19:21:46.004593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.940983 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 19:21:46.004593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.942541 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-20.ec2.internal" event="NodeHasSufficientMemory" Jul 13 19:21:46.004593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.942573 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-20.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 19:21:46.004593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.942584 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-20.ec2.internal" event="NodeHasSufficientPID" Jul 13 19:21:46.004593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.942617 2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-20.ec2.internal" Jul 13 19:21:46.004593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:45.952847 2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-20.ec2.internal" Jul 13 19:21:46.173585 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.172619 2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 19:21:46.173585 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.172862 2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 19:21:46.173585 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.172887 2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 19:21:46.173585 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.172955 2556 nodeinfomanager.go:417] Failed to publish CSINode: Post "https://ab486752fcdd945ec979b047ae357c7d-ab0e033fbd523808.elb.us-east-1.amazonaws.com:6443/apis/storage.k8s.io/v1/csinodes": read tcp 10.0.132.20:51962->3.225.72.118:6443: use of closed network connection Jul 13 19:21:46.173585 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:46.173042 2556 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: Post "https://ab486752fcdd945ec979b047ae357c7d-ab0e033fbd523808.elb.us-east-1.amazonaws.com:6443/apis/storage.k8s.io/v1/csinodes": read tcp 10.0.132.20:51962->3.225.72.118:6443: use of closed network connection Jul 13 19:21:46.747509 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.747387 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 19:16:45 +0000 UTC" deadline="2028-03-28 13:42:19.241673584 +0000 UTC" Jul 13 19:21:46.748023 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.747520 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14970h20m32.494171925s" Jul 13 19:21:46.748548 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.748535 2556 apiserver.go:52] "Watching apiserver" Jul 13 19:21:46.755169 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.755146 2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 19:21:46.767732 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.767689 2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 19:21:46.787284 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.787253 2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 19:21:46.820584 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.820548 2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-snqlr" Jul 13 19:21:46.830805 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.830768 2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-snqlr" Jul 13 19:21:46.841008 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.840974 2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 19:21:46.842374 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.842345 2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 19:21:46.842491 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.842389 2556 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 19:21:46.842491 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.842413 2556 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 19:21:46.842491 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:46.842453 2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 13 19:21:46.844782 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.844759 2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 19:21:46.944295 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.944174 2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-20.ec2.internal"] Jul 13 19:21:46.946078 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.946056 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal" Jul 13 19:21:46.947103 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.947072 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-wzrvz","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg","openshift-cluster-node-tuning-operator/tuned-862gg","openshift-multus/multus-additional-cni-plugins-lmdrb","openshift-multus/network-metrics-daemon-sgfvm","openshift-network-diagnostics/network-check-target-f2lcv","openshift-ovn-kubernetes/ovnkube-node-bk46s","openshift-dns/node-resolver-5fj4k","openshift-image-registry/node-ca-rkbds","openshift-multus/multus-4t2sn","openshift-network-operator/iptables-alerter-nlq97","kube-system/global-pull-secret-syncer-qgb5b"] Jul 13 19:21:46.947265 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.947199 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-20.ec2.internal" Jul 13 19:21:46.948322 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.948307 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wzrvz" Jul 13 19:21:46.949733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.949706 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:46.950820 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.950800 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:46.952101 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.952079 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:46.953355 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.953333 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:21:46.953456 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:46.953432 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:21:46.955351 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.954890 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:21:46.955351 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:46.954987 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:21:46.956075 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.955985 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 19:21:46.957002 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.956121 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-t9jw9\"" Jul 13 19:21:46.957002 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.956509 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 19:21:46.957002 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.956587 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 19:21:46.957002 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.956825 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 19:21:46.957002 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.956962 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 19:21:46.957297 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.957171 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 19:21:46.957297 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.957207 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 19:21:46.957483 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.957455 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-q87nl\"" Jul 13 19:21:46.957901 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.957493 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4scwj\"" Jul 13 19:21:46.957901 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.957637 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 19:21:46.957901 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.957645 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 19:21:46.958146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.957948 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 19:21:46.958146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.958022 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tgw8n\"" Jul 13 19:21:46.958579 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.958500 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.958579 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.958059 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 19:21:46.958808 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.958595 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 19:21:46.959944 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.959927 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5fj4k" Jul 13 19:21:46.961103 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.961087 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rkbds" Jul 13 19:21:46.962336 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.962321 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4t2sn" Jul 13 19:21:46.963545 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.963530 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nlq97" Jul 13 19:21:46.964563 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.964539 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 19:21:46.964765 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.964569 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 19:21:46.964765 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.964573 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 19:21:46.964765 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.964647 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 19:21:46.964765 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.964691 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:46.964765 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:46.964749 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:21:46.965138 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.964949 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 19:21:46.965138 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.964977 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 19:21:46.965138 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.965059 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 19:21:46.965272 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.965188 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 19:21:46.965272 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.965256 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 19:21:46.965378 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.965295 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 19:21:46.965378 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.965318 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 19:21:46.965378 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.965371 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7d5b6\"" Jul 13 19:21:46.965500 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.965449 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4gpmg\"" Jul 13 19:21:46.965568 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.965554 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zlklr\"" Jul 13 19:21:46.965628 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.965615 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f8cz8\"" Jul 13 19:21:46.965663 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.965618 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 19:21:46.966047 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.966032 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 19:21:46.966206 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.966189 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 19:21:46.966245 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.966223 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 19:21:46.966282 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.966272 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2q8pt\"" Jul 13 19:21:46.967589 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.967573 2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal" Jul 13 19:21:46.968094 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.968080 2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 19:21:46.973951 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.973930 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea52e435-228a-4124-89ee-5dfa0c6fa47f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:46.974028 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.973962 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnszl\" (UniqueName: \"kubernetes.io/projected/323bf873-3fca-4fa4-82c9-f2f3be457ed3-kube-api-access-cnszl\") pod \"iptables-alerter-nlq97\" (UID: \"323bf873-3fca-4fa4-82c9-f2f3be457ed3\") " pod="openshift-network-operator/iptables-alerter-nlq97" Jul 13 19:21:46.974028 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.973983 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-kubernetes\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:46.974028 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974001 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-run-ovn\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.974145 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974035 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret\") pod \"global-pull-secret-syncer-qgb5b\" (UID: \"5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4\") " pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:46.974145 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974053 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/84571915a3a9452655334a0c5a120bd4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal\" (UID: \"84571915a3a9452655334a0c5a120bd4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal" Jul 13 19:21:46.974145 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974071 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bd6bcf64-ab9b-4c5f-b23b-07f454be4c24-agent-certs\") pod \"konnectivity-agent-wzrvz\" (UID: \"bd6bcf64-ab9b-4c5f-b23b-07f454be4c24\") " pod="kube-system/konnectivity-agent-wzrvz" Jul 13 19:21:46.974145 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974087 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-sys\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:46.974145 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974103 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72f86323-f67b-49e2-b010-df5515861ddd-system-cni-dir\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:46.974145 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974122 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72f86323-f67b-49e2-b010-df5515861ddd-cni-binary-copy\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:46.974145 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974138 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-env-overrides\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.974429 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974153 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-system-cni-dir\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:46.974429 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974172 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-os-release\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:46.974429 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974189 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7gx7\" (UniqueName: \"kubernetes.io/projected/9b5b0a0b-15eb-4d6d-83cf-117a9009583b-kube-api-access-v7gx7\") pod \"node-resolver-5fj4k\" (UID: \"9b5b0a0b-15eb-4d6d-83cf-117a9009583b\") " pod="openshift-dns/node-resolver-5fj4k" Jul 13 19:21:46.974429 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974205 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c0a858d7-aa1f-4af2-934a-240c6ed35046-serviceca\") pod \"node-ca-rkbds\" (UID: \"c0a858d7-aa1f-4af2-934a-240c6ed35046\") " pod="openshift-image-registry/node-ca-rkbds" Jul 13 19:21:46.974429 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974237 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-multus-daemon-config\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:46.974429 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974254 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzqnt\" (UniqueName: \"kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt\") pod \"network-check-target-f2lcv\" (UID: \"83623552-aebf-463b-acc9-4cedaa7a7feb\") " pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:21:46.974429 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974281 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-host-var-lib-cni-multus\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:46.974429 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974314 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-sysconfig\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:46.974429 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974339 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-var-lib-kubelet\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:46.974429 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974368 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-cni-binary-copy\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:46.974429 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974385 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-host-run-k8s-cni-cncf-io\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:46.974429 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974403 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/323bf873-3fca-4fa4-82c9-f2f3be457ed3-iptables-alerter-script\") pod \"iptables-alerter-nlq97\" (UID: \"323bf873-3fca-4fa4-82c9-f2f3be457ed3\") " pod="openshift-network-operator/iptables-alerter-nlq97" Jul 13 19:21:46.975050 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974444 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-sysctl-conf\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:46.975050 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974460 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72f86323-f67b-49e2-b010-df5515861ddd-cnibin\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:46.975050 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974480 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g6dq\" (UniqueName: \"kubernetes.io/projected/6278283e-2c4b-4f66-915b-934fc03eb4bf-kube-api-access-9g6dq\") pod \"network-metrics-daemon-sgfvm\" (UID: \"6278283e-2c4b-4f66-915b-934fc03eb4bf\") " pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:21:46.975050 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974496 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-cnibin\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:46.975050 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974512 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-host-run-multus-certs\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:46.975050 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974530 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f88g9\" (UniqueName: \"kubernetes.io/projected/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-kube-api-access-f88g9\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:46.975050 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974563 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e49ce7abc689f72d6258a00fc408109c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-20.ec2.internal\" (UID: \"e49ce7abc689f72d6258a00fc408109c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-20.ec2.internal" Jul 13 19:21:46.975050 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974614 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-modprobe-d\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:46.975050 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974668 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-var-lib-openvswitch\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.975050 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974696 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c0a858d7-aa1f-4af2-934a-240c6ed35046-host\") pod \"node-ca-rkbds\" (UID: \"c0a858d7-aa1f-4af2-934a-240c6ed35046\") " pod="openshift-image-registry/node-ca-rkbds" Jul 13 19:21:46.975050 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974724 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea52e435-228a-4124-89ee-5dfa0c6fa47f-socket-dir\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:46.975050 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974755 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-multus-conf-dir\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:46.975050 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974786 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72f86323-f67b-49e2-b010-df5515861ddd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:46.975050 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974834 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-run-systemd\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.975595 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974858 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-log-socket\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.975595 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974882 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-run-ovn-kubernetes\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.975595 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974907 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-dbus\") pod \"global-pull-secret-syncer-qgb5b\" (UID: \"5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4\") " pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:46.975595 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974932 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea52e435-228a-4124-89ee-5dfa0c6fa47f-device-dir\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:46.975595 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974956 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-tuned\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:46.975595 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.974980 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12a6461b-fb8a-4d84-a0dc-707b552a14b6-tmp\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:46.975595 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975024 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-kubelet-config\") pod \"global-pull-secret-syncer-qgb5b\" (UID: \"5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4\") " pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:46.975595 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975059 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-run\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:46.975595 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975084 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9b5b0a0b-15eb-4d6d-83cf-117a9009583b-hosts-file\") pod \"node-resolver-5fj4k\" (UID: \"9b5b0a0b-15eb-4d6d-83cf-117a9009583b\") " pod="openshift-dns/node-resolver-5fj4k" Jul 13 19:21:46.975595 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975107 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea52e435-228a-4124-89ee-5dfa0c6fa47f-sys-fs\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:46.975595 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975129 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6qpn\" (UniqueName: \"kubernetes.io/projected/12a6461b-fb8a-4d84-a0dc-707b552a14b6-kube-api-access-p6qpn\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:46.975595 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975150 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-systemd-units\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.975595 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975174 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-cni-netd\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.975595 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975195 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpdq8\" (UniqueName: \"kubernetes.io/projected/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-kube-api-access-xpdq8\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.976322 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975214 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72f86323-f67b-49e2-b010-df5515861ddd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:46.976322 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975239 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bd6bcf64-ab9b-4c5f-b23b-07f454be4c24-konnectivity-ca\") pod \"konnectivity-agent-wzrvz\" (UID: \"bd6bcf64-ab9b-4c5f-b23b-07f454be4c24\") " pod="kube-system/konnectivity-agent-wzrvz" Jul 13 19:21:46.976322 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975266 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-host-run-netns\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:46.976322 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975315 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/72f86323-f67b-49e2-b010-df5515861ddd-os-release\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:46.976322 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975340 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-etc-openvswitch\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.976322 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975357 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-cni-bin\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.976322 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975374 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.976322 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975400 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b5b0a0b-15eb-4d6d-83cf-117a9009583b-tmp-dir\") pod \"node-resolver-5fj4k\" (UID: \"9b5b0a0b-15eb-4d6d-83cf-117a9009583b\") " pod="openshift-dns/node-resolver-5fj4k" Jul 13 19:21:46.976322 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975419 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea52e435-228a-4124-89ee-5dfa0c6fa47f-registration-dir\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:46.976322 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975435 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/323bf873-3fca-4fa4-82c9-f2f3be457ed3-host-slash\") pod \"iptables-alerter-nlq97\" (UID: \"323bf873-3fca-4fa4-82c9-f2f3be457ed3\") " pod="openshift-network-operator/iptables-alerter-nlq97" Jul 13 19:21:46.976322 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975476 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-host\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:46.976322 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975504 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-node-log\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.976322 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975533 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-ovn-node-metrics-cert\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.976702 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975560 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/84571915a3a9452655334a0c5a120bd4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal\" (UID: \"84571915a3a9452655334a0c5a120bd4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal" Jul 13 19:21:46.976702 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975587 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdffj\" (UniqueName: \"kubernetes.io/projected/72f86323-f67b-49e2-b010-df5515861ddd-kube-api-access-tdffj\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:46.976702 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975615 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs\") pod \"network-metrics-daemon-sgfvm\" (UID: \"6278283e-2c4b-4f66-915b-934fc03eb4bf\") " pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:21:46.976702 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975640 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-multus-cni-dir\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:46.976702 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975663 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-systemd\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:46.976702 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975692 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-lib-modules\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:46.976702 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975718 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72f86323-f67b-49e2-b010-df5515861ddd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:46.976702 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975743 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-kubelet\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.976702 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975767 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea52e435-228a-4124-89ee-5dfa0c6fa47f-etc-selinux\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:46.976702 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975792 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-hostroot\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:46.976702 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975837 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-sysctl-d\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:46.976702 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975862 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-run-openvswitch\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.976702 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975887 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-ovnkube-script-lib\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.977094 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975914 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k6bb\" (UniqueName: \"kubernetes.io/projected/c0a858d7-aa1f-4af2-934a-240c6ed35046-kube-api-access-4k6bb\") pod \"node-ca-rkbds\" (UID: \"c0a858d7-aa1f-4af2-934a-240c6ed35046\") " pod="openshift-image-registry/node-ca-rkbds" Jul 13 19:21:46.977094 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975940 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-host-var-lib-kubelet\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:46.977094 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.975967 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-etc-kubernetes\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:46.977094 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.976000 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqjp5\" (UniqueName: \"kubernetes.io/projected/ea52e435-228a-4124-89ee-5dfa0c6fa47f-kube-api-access-cqjp5\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:46.977094 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.976028 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-multus-socket-dir-parent\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:46.977094 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.976052 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-host-var-lib-cni-bin\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:46.977094 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.976079 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-slash\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.977094 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.976109 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-run-netns\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.977094 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.976134 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-ovnkube-config\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:46.985825 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.985791 2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-20.ec2.internal" Jul 13 19:21:46.990721 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.990704 2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 19:21:46.991426 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:46.991405 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal"] Jul 13 19:21:47.008043 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.008012 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-20.ec2.internal"] Jul 13 19:21:47.009108 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.009091 2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 19:21:47.042961 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.042917 2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 19:21:47.046280 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.046254 2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 19:21:47.076396 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076365 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-run\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.076525 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076401 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9b5b0a0b-15eb-4d6d-83cf-117a9009583b-hosts-file\") pod \"node-resolver-5fj4k\" (UID: \"9b5b0a0b-15eb-4d6d-83cf-117a9009583b\") " pod="openshift-dns/node-resolver-5fj4k" Jul 13 19:21:47.076525 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076427 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea52e435-228a-4124-89ee-5dfa0c6fa47f-sys-fs\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:47.076525 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076451 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6qpn\" (UniqueName: \"kubernetes.io/projected/12a6461b-fb8a-4d84-a0dc-707b552a14b6-kube-api-access-p6qpn\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.076525 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076477 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-systemd-units\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.076525 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076500 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-cni-netd\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.076525 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076505 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9b5b0a0b-15eb-4d6d-83cf-117a9009583b-hosts-file\") pod \"node-resolver-5fj4k\" (UID: \"9b5b0a0b-15eb-4d6d-83cf-117a9009583b\") " pod="openshift-dns/node-resolver-5fj4k" Jul 13 19:21:47.076525 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076527 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpdq8\" (UniqueName: \"kubernetes.io/projected/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-kube-api-access-xpdq8\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.076881 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076539 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-run\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.076881 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076588 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-systemd-units\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.076881 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076612 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea52e435-228a-4124-89ee-5dfa0c6fa47f-sys-fs\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:47.076881 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076551 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72f86323-f67b-49e2-b010-df5515861ddd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:47.076881 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076651 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bd6bcf64-ab9b-4c5f-b23b-07f454be4c24-konnectivity-ca\") pod \"konnectivity-agent-wzrvz\" (UID: \"bd6bcf64-ab9b-4c5f-b23b-07f454be4c24\") " pod="kube-system/konnectivity-agent-wzrvz" Jul 13 19:21:47.076881 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076648 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-cni-netd\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.076881 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076692 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-host-run-netns\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.076881 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076721 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/72f86323-f67b-49e2-b010-df5515861ddd-os-release\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:47.076881 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076745 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-etc-openvswitch\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.076881 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076751 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-host-run-netns\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.076881 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076768 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-cni-bin\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.076881 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076811 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.076881 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076841 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b5b0a0b-15eb-4d6d-83cf-117a9009583b-tmp-dir\") pod \"node-resolver-5fj4k\" (UID: \"9b5b0a0b-15eb-4d6d-83cf-117a9009583b\") " pod="openshift-dns/node-resolver-5fj4k" Jul 13 19:21:47.076881 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076840 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-etc-openvswitch\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.077464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076856 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/72f86323-f67b-49e2-b010-df5515861ddd-os-release\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:47.077464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076867 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea52e435-228a-4124-89ee-5dfa0c6fa47f-registration-dir\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:47.077464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076857 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-cni-bin\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.077464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076894 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.077464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076939 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/323bf873-3fca-4fa4-82c9-f2f3be457ed3-host-slash\") pod \"iptables-alerter-nlq97\" (UID: \"323bf873-3fca-4fa4-82c9-f2f3be457ed3\") " pod="openshift-network-operator/iptables-alerter-nlq97" Jul 13 19:21:47.077464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.076976 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-host\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.077464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077012 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-node-log\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.077464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077021 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/323bf873-3fca-4fa4-82c9-f2f3be457ed3-host-slash\") pod \"iptables-alerter-nlq97\" (UID: \"323bf873-3fca-4fa4-82c9-f2f3be457ed3\") " pod="openshift-network-operator/iptables-alerter-nlq97" Jul 13 19:21:47.077464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077040 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-ovn-node-metrics-cert\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.077464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077040 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-host\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.077464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077065 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-node-log\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.077464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077029 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea52e435-228a-4124-89ee-5dfa0c6fa47f-registration-dir\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:47.077464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077079 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/84571915a3a9452655334a0c5a120bd4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal\" (UID: \"84571915a3a9452655334a0c5a120bd4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal" Jul 13 19:21:47.077464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077114 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdffj\" (UniqueName: \"kubernetes.io/projected/72f86323-f67b-49e2-b010-df5515861ddd-kube-api-access-tdffj\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:47.078146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077144 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs\") pod \"network-metrics-daemon-sgfvm\" (UID: \"6278283e-2c4b-4f66-915b-934fc03eb4bf\") " pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:21:47.078146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077172 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/84571915a3a9452655334a0c5a120bd4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal\" (UID: \"84571915a3a9452655334a0c5a120bd4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal" Jul 13 19:21:47.078146 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:47.077248 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:21:47.078146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077268 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-multus-cni-dir\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.078146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077289 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-systemd\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.078146 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:47.077328 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs podName:6278283e-2c4b-4f66-915b-934fc03eb4bf nodeName:}" failed. No retries permitted until 2026-07-13 19:21:47.577300218 +0000 UTC m=+2.691207915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs") pod "network-metrics-daemon-sgfvm" (UID: "6278283e-2c4b-4f66-915b-934fc03eb4bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:21:47.078146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077337 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-systemd\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.078146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077358 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-lib-modules\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.078146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077394 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72f86323-f67b-49e2-b010-df5515861ddd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:47.078146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077421 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-kubelet\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.078146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077422 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-multus-cni-dir\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.078146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077506 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea52e435-228a-4124-89ee-5dfa0c6fa47f-etc-selinux\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:47.078146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077518 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-lib-modules\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.078146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077534 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-hostroot\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.078817 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077561 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-sysctl-d\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.078817 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077588 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-run-openvswitch\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.078817 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077535 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-kubelet\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.078817 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077615 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-ovnkube-script-lib\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.078817 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077628 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-hostroot\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.078817 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077635 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea52e435-228a-4124-89ee-5dfa0c6fa47f-etc-selinux\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:47.078817 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077646 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k6bb\" (UniqueName: \"kubernetes.io/projected/c0a858d7-aa1f-4af2-934a-240c6ed35046-kube-api-access-4k6bb\") pod \"node-ca-rkbds\" (UID: \"c0a858d7-aa1f-4af2-934a-240c6ed35046\") " pod="openshift-image-registry/node-ca-rkbds" Jul 13 19:21:47.078817 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077659 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-run-openvswitch\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.078817 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077672 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-host-var-lib-kubelet\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.078817 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077703 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-etc-kubernetes\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.078817 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077731 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqjp5\" (UniqueName: \"kubernetes.io/projected/ea52e435-228a-4124-89ee-5dfa0c6fa47f-kube-api-access-cqjp5\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:47.078817 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077739 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-sysctl-d\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.078817 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077761 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-multus-socket-dir-parent\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.078817 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077794 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-etc-kubernetes\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.078817 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077796 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-host-var-lib-cni-bin\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.079593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077867 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-slash\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.079593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077895 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-run-netns\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.079593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077946 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-host-var-lib-kubelet\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.079593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077944 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-ovnkube-config\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.079593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077968 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-host-var-lib-cni-bin\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.079593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.077989 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea52e435-228a-4124-89ee-5dfa0c6fa47f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:47.079593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078009 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnszl\" (UniqueName: \"kubernetes.io/projected/323bf873-3fca-4fa4-82c9-f2f3be457ed3-kube-api-access-cnszl\") pod \"iptables-alerter-nlq97\" (UID: \"323bf873-3fca-4fa4-82c9-f2f3be457ed3\") " pod="openshift-network-operator/iptables-alerter-nlq97" Jul 13 19:21:47.079593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078031 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-kubernetes\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.079593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078058 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-run-ovn\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.079593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078086 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret\") pod \"global-pull-secret-syncer-qgb5b\" (UID: \"5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4\") " pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:47.079593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078117 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/84571915a3a9452655334a0c5a120bd4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal\" (UID: \"84571915a3a9452655334a0c5a120bd4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal" Jul 13 19:21:47.079593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078149 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bd6bcf64-ab9b-4c5f-b23b-07f454be4c24-agent-certs\") pod \"konnectivity-agent-wzrvz\" (UID: \"bd6bcf64-ab9b-4c5f-b23b-07f454be4c24\") " pod="kube-system/konnectivity-agent-wzrvz" Jul 13 19:21:47.079593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078152 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-slash\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.079593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078178 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-sys\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.080288 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078206 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72f86323-f67b-49e2-b010-df5515861ddd-system-cni-dir\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:47.080288 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078237 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72f86323-f67b-49e2-b010-df5515861ddd-cni-binary-copy\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:47.080288 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078243 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-kubernetes\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.080288 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:47.078262 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 19:21:47.080288 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078279 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-env-overrides\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.080288 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:47.078319 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret podName:5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4 nodeName:}" failed. No retries permitted until 2026-07-13 19:21:47.578298727 +0000 UTC m=+2.692206435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret") pod "global-pull-secret-syncer-qgb5b" (UID: "5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4") : object "kube-system"/"original-pull-secret" not registered Jul 13 19:21:47.080288 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078326 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-run-netns\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.080288 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078339 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-system-cni-dir\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.080288 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078403 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-sys\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.080288 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078408 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/84571915a3a9452655334a0c5a120bd4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal\" (UID: \"84571915a3a9452655334a0c5a120bd4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal" Jul 13 19:21:47.080288 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078029 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-multus-socket-dir-parent\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.080288 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078460 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-run-ovn\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.080288 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078610 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea52e435-228a-4124-89ee-5dfa0c6fa47f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:47.080288 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078617 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-os-release\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.080935 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078653 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7gx7\" (UniqueName: \"kubernetes.io/projected/9b5b0a0b-15eb-4d6d-83cf-117a9009583b-kube-api-access-v7gx7\") pod \"node-resolver-5fj4k\" (UID: \"9b5b0a0b-15eb-4d6d-83cf-117a9009583b\") " pod="openshift-dns/node-resolver-5fj4k" Jul 13 19:21:47.080935 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078661 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72f86323-f67b-49e2-b010-df5515861ddd-system-cni-dir\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:47.080935 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078680 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c0a858d7-aa1f-4af2-934a-240c6ed35046-serviceca\") pod \"node-ca-rkbds\" (UID: \"c0a858d7-aa1f-4af2-934a-240c6ed35046\") " pod="openshift-image-registry/node-ca-rkbds" Jul 13 19:21:47.080935 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078708 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-multus-daemon-config\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.080935 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078731 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-os-release\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.080935 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078741 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzqnt\" (UniqueName: \"kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt\") pod \"network-check-target-f2lcv\" (UID: \"83623552-aebf-463b-acc9-4cedaa7a7feb\") " pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:21:47.080935 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078770 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-host-var-lib-cni-multus\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.080935 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078772 2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 19:21:47.080935 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078798 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-sysconfig\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.080935 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078823 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-var-lib-kubelet\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.080935 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078865 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-cni-binary-copy\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.080935 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078895 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-host-run-k8s-cni-cncf-io\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.080935 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078905 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-ovnkube-script-lib\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.080935 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078923 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/323bf873-3fca-4fa4-82c9-f2f3be457ed3-iptables-alerter-script\") pod \"iptables-alerter-nlq97\" (UID: \"323bf873-3fca-4fa4-82c9-f2f3be457ed3\") " pod="openshift-network-operator/iptables-alerter-nlq97" Jul 13 19:21:47.080935 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078952 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-sysctl-conf\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.081694 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078980 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72f86323-f67b-49e2-b010-df5515861ddd-cnibin\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:47.081694 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078986 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bd6bcf64-ab9b-4c5f-b23b-07f454be4c24-konnectivity-ca\") pod \"konnectivity-agent-wzrvz\" (UID: \"bd6bcf64-ab9b-4c5f-b23b-07f454be4c24\") " pod="kube-system/konnectivity-agent-wzrvz" Jul 13 19:21:47.081694 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079027 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g6dq\" (UniqueName: \"kubernetes.io/projected/6278283e-2c4b-4f66-915b-934fc03eb4bf-kube-api-access-9g6dq\") pod \"network-metrics-daemon-sgfvm\" (UID: \"6278283e-2c4b-4f66-915b-934fc03eb4bf\") " pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:21:47.081694 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079034 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-sysconfig\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.081694 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079063 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-var-lib-kubelet\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.081694 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079069 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-cnibin\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.081694 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079111 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72f86323-f67b-49e2-b010-df5515861ddd-cnibin\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:47.081694 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079119 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-host-run-multus-certs\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.081694 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079152 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-host-run-k8s-cni-cncf-io\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.081694 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079154 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f88g9\" (UniqueName: \"kubernetes.io/projected/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-kube-api-access-f88g9\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.081694 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079160 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-sysctl-conf\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.081694 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079181 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e49ce7abc689f72d6258a00fc408109c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-20.ec2.internal\" (UID: \"e49ce7abc689f72d6258a00fc408109c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-20.ec2.internal" Jul 13 19:21:47.081694 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079205 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-cnibin\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.081694 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079210 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-modprobe-d\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.081694 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079241 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-var-lib-openvswitch\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.082480 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079271 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c0a858d7-aa1f-4af2-934a-240c6ed35046-host\") pod \"node-ca-rkbds\" (UID: \"c0a858d7-aa1f-4af2-934a-240c6ed35046\") " pod="openshift-image-registry/node-ca-rkbds" Jul 13 19:21:47.082480 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079300 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea52e435-228a-4124-89ee-5dfa0c6fa47f-socket-dir\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:47.082480 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079327 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-multus-conf-dir\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.082480 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079354 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72f86323-f67b-49e2-b010-df5515861ddd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:47.082480 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079383 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-run-systemd\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.082480 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079412 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-log-socket\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.082480 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079426 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-multus-daemon-config\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.082480 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079440 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-run-ovn-kubernetes\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.082480 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079470 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-dbus\") pod \"global-pull-secret-syncer-qgb5b\" (UID: \"5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4\") " pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:47.082480 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079488 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-multus-conf-dir\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.082480 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079497 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea52e435-228a-4124-89ee-5dfa0c6fa47f-device-dir\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:47.082480 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079523 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-tuned\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.082480 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079551 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12a6461b-fb8a-4d84-a0dc-707b552a14b6-tmp\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.082480 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079557 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c0a858d7-aa1f-4af2-934a-240c6ed35046-host\") pod \"node-ca-rkbds\" (UID: \"c0a858d7-aa1f-4af2-934a-240c6ed35046\") " pod="openshift-image-registry/node-ca-rkbds" Jul 13 19:21:47.082480 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079580 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-kubelet-config\") pod \"global-pull-secret-syncer-qgb5b\" (UID: \"5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4\") " pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:47.083172 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079657 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea52e435-228a-4124-89ee-5dfa0c6fa47f-socket-dir\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:47.083172 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079819 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72f86323-f67b-49e2-b010-df5515861ddd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:47.083172 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079905 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-run-systemd\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.083172 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079925 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/323bf873-3fca-4fa4-82c9-f2f3be457ed3-iptables-alerter-script\") pod \"iptables-alerter-nlq97\" (UID: \"323bf873-3fca-4fa4-82c9-f2f3be457ed3\") " pod="openshift-network-operator/iptables-alerter-nlq97" Jul 13 19:21:47.083172 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079965 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-log-socket\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.083172 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079966 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c0a858d7-aa1f-4af2-934a-240c6ed35046-serviceca\") pod \"node-ca-rkbds\" (UID: \"c0a858d7-aa1f-4af2-934a-240c6ed35046\") " pod="openshift-image-registry/node-ca-rkbds" Jul 13 19:21:47.083172 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.078708 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b5b0a0b-15eb-4d6d-83cf-117a9009583b-tmp-dir\") pod \"node-resolver-5fj4k\" (UID: \"9b5b0a0b-15eb-4d6d-83cf-117a9009583b\") " pod="openshift-dns/node-resolver-5fj4k" Jul 13 19:21:47.083172 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079998 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-cni-binary-copy\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.083172 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.080011 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-host-var-lib-cni-multus\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.083172 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.080062 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-host-run-ovn-kubernetes\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.083172 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.080065 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-host-run-multus-certs\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.083172 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.080079 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72f86323-f67b-49e2-b010-df5515861ddd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:47.083172 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.080104 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-ovnkube-config\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.083172 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.080117 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e49ce7abc689f72d6258a00fc408109c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-20.ec2.internal\" (UID: \"e49ce7abc689f72d6258a00fc408109c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-20.ec2.internal" Jul 13 19:21:47.083172 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.079526 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-var-lib-openvswitch\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.084033 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.080135 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-dbus\") pod \"global-pull-secret-syncer-qgb5b\" (UID: \"5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4\") " pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:47.084033 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.080160 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-system-cni-dir\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.084033 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.080182 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea52e435-228a-4124-89ee-5dfa0c6fa47f-device-dir\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:47.084033 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.080206 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-modprobe-d\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.084033 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.080219 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-kubelet-config\") pod \"global-pull-secret-syncer-qgb5b\" (UID: \"5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4\") " pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:47.084033 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.080657 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72f86323-f67b-49e2-b010-df5515861ddd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:47.084033 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.080690 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72f86323-f67b-49e2-b010-df5515861ddd-cni-binary-copy\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:47.084033 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.081249 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-env-overrides\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.084033 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.083200 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/12a6461b-fb8a-4d84-a0dc-707b552a14b6-etc-tuned\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.084033 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.083293 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12a6461b-fb8a-4d84-a0dc-707b552a14b6-tmp\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.084033 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.083388 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-ovn-node-metrics-cert\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.084033 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.083443 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bd6bcf64-ab9b-4c5f-b23b-07f454be4c24-agent-certs\") pod \"konnectivity-agent-wzrvz\" (UID: \"bd6bcf64-ab9b-4c5f-b23b-07f454be4c24\") " pod="kube-system/konnectivity-agent-wzrvz" Jul 13 19:21:47.108984 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.108956 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6qpn\" (UniqueName: \"kubernetes.io/projected/12a6461b-fb8a-4d84-a0dc-707b552a14b6-kube-api-access-p6qpn\") pod \"tuned-862gg\" (UID: \"12a6461b-fb8a-4d84-a0dc-707b552a14b6\") " pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.125631 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:47.125595 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 19:21:47.125631 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:47.125631 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 19:21:47.125903 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:47.125643 2556 projected.go:196] Error preparing data for projected volume kube-api-access-rzqnt for pod openshift-network-diagnostics/network-check-target-f2lcv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:21:47.125903 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:47.125717 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt podName:83623552-aebf-463b-acc9-4cedaa7a7feb nodeName:}" failed. No retries permitted until 2026-07-13 19:21:47.625697311 +0000 UTC m=+2.739605024 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rzqnt" (UniqueName: "kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt") pod "network-check-target-f2lcv" (UID: "83623552-aebf-463b-acc9-4cedaa7a7feb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:21:47.128533 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.128500 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqjp5\" (UniqueName: \"kubernetes.io/projected/ea52e435-228a-4124-89ee-5dfa0c6fa47f-kube-api-access-cqjp5\") pod \"aws-ebs-csi-driver-node-jckhg\" (UID: \"ea52e435-228a-4124-89ee-5dfa0c6fa47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:47.128654 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.128507 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdffj\" (UniqueName: \"kubernetes.io/projected/72f86323-f67b-49e2-b010-df5515861ddd-kube-api-access-tdffj\") pod \"multus-additional-cni-plugins-lmdrb\" (UID: \"72f86323-f67b-49e2-b010-df5515861ddd\") " pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:47.134268 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.134239 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnszl\" (UniqueName: \"kubernetes.io/projected/323bf873-3fca-4fa4-82c9-f2f3be457ed3-kube-api-access-cnszl\") pod \"iptables-alerter-nlq97\" (UID: \"323bf873-3fca-4fa4-82c9-f2f3be457ed3\") " pod="openshift-network-operator/iptables-alerter-nlq97" Jul 13 19:21:47.134423 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.134363 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k6bb\" (UniqueName: \"kubernetes.io/projected/c0a858d7-aa1f-4af2-934a-240c6ed35046-kube-api-access-4k6bb\") pod \"node-ca-rkbds\" (UID: \"c0a858d7-aa1f-4af2-934a-240c6ed35046\") " pod="openshift-image-registry/node-ca-rkbds" Jul 13 19:21:47.134423 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.134369 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpdq8\" (UniqueName: \"kubernetes.io/projected/a72fd793-62c4-49b9-b3cb-06d08aa88ec7-kube-api-access-xpdq8\") pod \"ovnkube-node-bk46s\" (UID: \"a72fd793-62c4-49b9-b3cb-06d08aa88ec7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.135034 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.135013 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7gx7\" (UniqueName: \"kubernetes.io/projected/9b5b0a0b-15eb-4d6d-83cf-117a9009583b-kube-api-access-v7gx7\") pod \"node-resolver-5fj4k\" (UID: \"9b5b0a0b-15eb-4d6d-83cf-117a9009583b\") " pod="openshift-dns/node-resolver-5fj4k" Jul 13 19:21:47.138336 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.138308 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f88g9\" (UniqueName: \"kubernetes.io/projected/bb78901a-bfbf-4f49-b2a2-2faf6ed439ab-kube-api-access-f88g9\") pod \"multus-4t2sn\" (UID: \"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab\") " pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.143151 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.143130 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g6dq\" (UniqueName: \"kubernetes.io/projected/6278283e-2c4b-4f66-915b-934fc03eb4bf-kube-api-access-9g6dq\") pod \"network-metrics-daemon-sgfvm\" (UID: \"6278283e-2c4b-4f66-915b-934fc03eb4bf\") " pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:21:47.294096 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.293990 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal" Jul 13 19:21:47.306524 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.306493 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wzrvz" Jul 13 19:21:47.311689 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.311664 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-20.ec2.internal" Jul 13 19:21:47.311810 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.311695 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" Jul 13 19:21:47.341528 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.341491 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-862gg" Jul 13 19:21:47.346372 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.346350 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lmdrb" Jul 13 19:21:47.367567 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.367540 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:21:47.379420 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.379390 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5fj4k" Jul 13 19:21:47.393519 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.393493 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rkbds" Jul 13 19:21:47.423631 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.423588 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4t2sn" Jul 13 19:21:47.429805 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.429767 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nlq97" Jul 13 19:21:47.584579 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.584474 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs\") pod \"network-metrics-daemon-sgfvm\" (UID: \"6278283e-2c4b-4f66-915b-934fc03eb4bf\") " pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:21:47.584579 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.584533 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret\") pod \"global-pull-secret-syncer-qgb5b\" (UID: \"5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4\") " pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:47.584855 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:47.584646 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:21:47.584855 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:47.584707 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs podName:6278283e-2c4b-4f66-915b-934fc03eb4bf nodeName:}" failed. No retries permitted until 2026-07-13 19:21:48.584692694 +0000 UTC m=+3.698600385 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs") pod "network-metrics-daemon-sgfvm" (UID: "6278283e-2c4b-4f66-915b-934fc03eb4bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:21:47.584855 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:47.584651 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 19:21:47.584855 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:47.584768 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret podName:5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4 nodeName:}" failed. No retries permitted until 2026-07-13 19:21:48.584757055 +0000 UTC m=+3.698664743 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret") pod "global-pull-secret-syncer-qgb5b" (UID: "5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4") : object "kube-system"/"original-pull-secret" not registered Jul 13 19:21:47.685249 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.685217 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzqnt\" (UniqueName: \"kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt\") pod \"network-check-target-f2lcv\" (UID: \"83623552-aebf-463b-acc9-4cedaa7a7feb\") " pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:21:47.685407 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:47.685381 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 19:21:47.685407 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:47.685401 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 19:21:47.685494 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:47.685413 2556 projected.go:196] Error preparing data for projected volume kube-api-access-rzqnt for pod openshift-network-diagnostics/network-check-target-f2lcv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:21:47.685494 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:47.685470 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt podName:83623552-aebf-463b-acc9-4cedaa7a7feb nodeName:}" failed. No retries permitted until 2026-07-13 19:21:48.685451231 +0000 UTC m=+3.799358921 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rzqnt" (UniqueName: "kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt") pod "network-check-target-f2lcv" (UID: "83623552-aebf-463b-acc9-4cedaa7a7feb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:21:47.833394 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.833325 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 19:16:46 +0000 UTC" deadline="2028-04-05 12:25:15.27431416 +0000 UTC" Jul 13 19:21:47.833394 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.833387 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15161h3m27.440929817s" Jul 13 19:21:47.959233 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:47.959185 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod323bf873_3fca_4fa4_82c9_f2f3be457ed3.slice/crio-6aa2897e9c148a7e79ed3119371ab41738d705ca7a96097a4f87951e70b3cbd1 WatchSource:0}: Error finding container 6aa2897e9c148a7e79ed3119371ab41738d705ca7a96097a4f87951e70b3cbd1: Status 404 returned error can't find the container with id 6aa2897e9c148a7e79ed3119371ab41738d705ca7a96097a4f87951e70b3cbd1 Jul 13 19:21:47.959501 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:47.959480 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b5b0a0b_15eb_4d6d_83cf_117a9009583b.slice/crio-214f3d3030be7b5cae49286e2fe83a7929750e5be482a77d86ee6cef9c7bfe0d WatchSource:0}: Error finding container 214f3d3030be7b5cae49286e2fe83a7929750e5be482a77d86ee6cef9c7bfe0d: Status 404 returned error can't find the container with id 214f3d3030be7b5cae49286e2fe83a7929750e5be482a77d86ee6cef9c7bfe0d Jul 13 19:21:47.965708 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:47.965688 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 19:21:47.970083 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:47.970056 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd6bcf64_ab9b_4c5f_b23b_07f454be4c24.slice/crio-3380a6c6853cedee27bd4564714801c01ed4a4fe6795494495598848e0d5358a WatchSource:0}: Error finding container 3380a6c6853cedee27bd4564714801c01ed4a4fe6795494495598848e0d5358a: Status 404 returned error can't find the container with id 3380a6c6853cedee27bd4564714801c01ed4a4fe6795494495598848e0d5358a Jul 13 19:21:47.999607 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:47.999577 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea52e435_228a_4124_89ee_5dfa0c6fa47f.slice/crio-75bbdefa0fc7524025b82d215d088f7eaa5af7a619cb23551aa6fd202dba2554 WatchSource:0}: Error finding container 75bbdefa0fc7524025b82d215d088f7eaa5af7a619cb23551aa6fd202dba2554: Status 404 returned error can't find the container with id 75bbdefa0fc7524025b82d215d088f7eaa5af7a619cb23551aa6fd202dba2554 Jul 13 19:21:48.000069 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:48.000044 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode49ce7abc689f72d6258a00fc408109c.slice/crio-1bb0a5d44f11ffa6e1a5bd9e73576c97e668867724139a905917496c1c65d41c WatchSource:0}: Error finding container 1bb0a5d44f11ffa6e1a5bd9e73576c97e668867724139a905917496c1c65d41c: Status 404 returned error can't find the container with id 1bb0a5d44f11ffa6e1a5bd9e73576c97e668867724139a905917496c1c65d41c Jul 13 19:21:48.016382 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:48.016343 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72f86323_f67b_49e2_b010_df5515861ddd.slice/crio-c8ae87c4076c1660edfa526eea7f63c54f940ab6a598b4b6be1ea37fbc28511c WatchSource:0}: Error finding container c8ae87c4076c1660edfa526eea7f63c54f940ab6a598b4b6be1ea37fbc28511c: Status 404 returned error can't find the container with id c8ae87c4076c1660edfa526eea7f63c54f940ab6a598b4b6be1ea37fbc28511c Jul 13 19:21:48.033730 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:48.033697 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda72fd793_62c4_49b9_b3cb_06d08aa88ec7.slice/crio-46a9c82db995ed4ac08a01cf057b7d29f243d164835fa327951b34cddee8ded7 WatchSource:0}: Error finding container 46a9c82db995ed4ac08a01cf057b7d29f243d164835fa327951b34cddee8ded7: Status 404 returned error can't find the container with id 46a9c82db995ed4ac08a01cf057b7d29f243d164835fa327951b34cddee8ded7 Jul 13 19:21:48.048287 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:48.048258 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0a858d7_aa1f_4af2_934a_240c6ed35046.slice/crio-ef2f5f4e5e1e37a26c709dc32fba13813d16120b80e13eee2e820348d3eacd2b WatchSource:0}: Error finding container ef2f5f4e5e1e37a26c709dc32fba13813d16120b80e13eee2e820348d3eacd2b: Status 404 returned error can't find the container with id ef2f5f4e5e1e37a26c709dc32fba13813d16120b80e13eee2e820348d3eacd2b Jul 13 19:21:48.065048 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:48.065019 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84571915a3a9452655334a0c5a120bd4.slice/crio-39ad61f7b9fb659194ef000150bf586425caed710a0c1590f2aad40afa73fcef WatchSource:0}: Error finding container 39ad61f7b9fb659194ef000150bf586425caed710a0c1590f2aad40afa73fcef: Status 404 returned error can't find the container with id 39ad61f7b9fb659194ef000150bf586425caed710a0c1590f2aad40afa73fcef Jul 13 19:21:48.202412 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:48.202380 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12a6461b_fb8a_4d84_a0dc_707b552a14b6.slice/crio-8f3cacbb5c1bd10c7e56a3088bd56d1964e0536da785cfea96b1e9017fc87b70 WatchSource:0}: Error finding container 8f3cacbb5c1bd10c7e56a3088bd56d1964e0536da785cfea96b1e9017fc87b70: Status 404 returned error can't find the container with id 8f3cacbb5c1bd10c7e56a3088bd56d1964e0536da785cfea96b1e9017fc87b70 Jul 13 19:21:48.283296 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:21:48.283259 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb78901a_bfbf_4f49_b2a2_2faf6ed439ab.slice/crio-3a87647f5a3eaa65b8625778a490059670f8f572f99a9c497a1467149ba09a0a WatchSource:0}: Error finding container 3a87647f5a3eaa65b8625778a490059670f8f572f99a9c497a1467149ba09a0a: Status 404 returned error can't find the container with id 3a87647f5a3eaa65b8625778a490059670f8f572f99a9c497a1467149ba09a0a Jul 13 19:21:48.592984 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:48.592801 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs\") pod \"network-metrics-daemon-sgfvm\" (UID: \"6278283e-2c4b-4f66-915b-934fc03eb4bf\") " pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:21:48.592984 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:48.592886 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret\") pod \"global-pull-secret-syncer-qgb5b\" (UID: \"5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4\") " pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:48.593251 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:48.593009 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:21:48.593251 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:48.593048 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 19:21:48.593251 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:48.593093 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs podName:6278283e-2c4b-4f66-915b-934fc03eb4bf nodeName:}" failed. No retries permitted until 2026-07-13 19:21:50.593070026 +0000 UTC m=+5.706977717 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs") pod "network-metrics-daemon-sgfvm" (UID: "6278283e-2c4b-4f66-915b-934fc03eb4bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:21:48.593251 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:48.593116 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret podName:5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4 nodeName:}" failed. No retries permitted until 2026-07-13 19:21:50.593105755 +0000 UTC m=+5.707013449 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret") pod "global-pull-secret-syncer-qgb5b" (UID: "5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4") : object "kube-system"/"original-pull-secret" not registered Jul 13 19:21:48.693789 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:48.693733 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzqnt\" (UniqueName: \"kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt\") pod \"network-check-target-f2lcv\" (UID: \"83623552-aebf-463b-acc9-4cedaa7a7feb\") " pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:21:48.693991 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:48.693939 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 19:21:48.693991 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:48.693965 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 19:21:48.693991 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:48.693978 2556 projected.go:196] Error preparing data for projected volume kube-api-access-rzqnt for pod openshift-network-diagnostics/network-check-target-f2lcv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:21:48.694407 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:48.694378 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt podName:83623552-aebf-463b-acc9-4cedaa7a7feb nodeName:}" failed. No retries permitted until 2026-07-13 19:21:50.694021067 +0000 UTC m=+5.807928772 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rzqnt" (UniqueName: "kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt") pod "network-check-target-f2lcv" (UID: "83623552-aebf-463b-acc9-4cedaa7a7feb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:21:48.833993 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:48.833559 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 19:16:46 +0000 UTC" deadline="2027-12-17 15:47:42.327507016 +0000 UTC" Jul 13 19:21:48.833993 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:48.833620 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12524h25m53.493891045s" Jul 13 19:21:48.845608 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:48.845572 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:21:48.845817 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:48.845758 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:21:48.846240 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:48.846213 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:21:48.846342 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:48.846326 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:21:48.846413 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:48.846387 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:48.846810 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:48.846462 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:21:48.871062 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:48.870997 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal" event={"ID":"84571915a3a9452655334a0c5a120bd4","Type":"ContainerStarted","Data":"39ad61f7b9fb659194ef000150bf586425caed710a0c1590f2aad40afa73fcef"} Jul 13 19:21:48.881086 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:48.881034 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" event={"ID":"a72fd793-62c4-49b9-b3cb-06d08aa88ec7","Type":"ContainerStarted","Data":"46a9c82db995ed4ac08a01cf057b7d29f243d164835fa327951b34cddee8ded7"} Jul 13 19:21:48.899404 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:48.899357 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" event={"ID":"ea52e435-228a-4124-89ee-5dfa0c6fa47f","Type":"ContainerStarted","Data":"75bbdefa0fc7524025b82d215d088f7eaa5af7a619cb23551aa6fd202dba2554"} Jul 13 19:21:48.906786 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:48.906726 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4t2sn" event={"ID":"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab","Type":"ContainerStarted","Data":"3a87647f5a3eaa65b8625778a490059670f8f572f99a9c497a1467149ba09a0a"} Jul 13 19:21:48.910844 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:48.910808 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-862gg" event={"ID":"12a6461b-fb8a-4d84-a0dc-707b552a14b6","Type":"ContainerStarted","Data":"8f3cacbb5c1bd10c7e56a3088bd56d1964e0536da785cfea96b1e9017fc87b70"} Jul 13 19:21:48.942201 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:48.942158 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rkbds" event={"ID":"c0a858d7-aa1f-4af2-934a-240c6ed35046","Type":"ContainerStarted","Data":"ef2f5f4e5e1e37a26c709dc32fba13813d16120b80e13eee2e820348d3eacd2b"} Jul 13 19:21:48.950517 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:48.950461 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmdrb" event={"ID":"72f86323-f67b-49e2-b010-df5515861ddd","Type":"ContainerStarted","Data":"c8ae87c4076c1660edfa526eea7f63c54f940ab6a598b4b6be1ea37fbc28511c"} Jul 13 19:21:48.969656 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:48.969609 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-20.ec2.internal" event={"ID":"e49ce7abc689f72d6258a00fc408109c","Type":"ContainerStarted","Data":"1bb0a5d44f11ffa6e1a5bd9e73576c97e668867724139a905917496c1c65d41c"} Jul 13 19:21:48.978499 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:48.978452 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wzrvz" event={"ID":"bd6bcf64-ab9b-4c5f-b23b-07f454be4c24","Type":"ContainerStarted","Data":"3380a6c6853cedee27bd4564714801c01ed4a4fe6795494495598848e0d5358a"} Jul 13 19:21:48.999456 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:48.999250 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nlq97" event={"ID":"323bf873-3fca-4fa4-82c9-f2f3be457ed3","Type":"ContainerStarted","Data":"6aa2897e9c148a7e79ed3119371ab41738d705ca7a96097a4f87951e70b3cbd1"} Jul 13 19:21:49.030942 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:49.030889 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5fj4k" event={"ID":"9b5b0a0b-15eb-4d6d-83cf-117a9009583b","Type":"ContainerStarted","Data":"214f3d3030be7b5cae49286e2fe83a7929750e5be482a77d86ee6cef9c7bfe0d"} Jul 13 19:21:50.615148 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:50.615104 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret\") pod \"global-pull-secret-syncer-qgb5b\" (UID: \"5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4\") " pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:50.615641 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:50.615217 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs\") pod \"network-metrics-daemon-sgfvm\" (UID: \"6278283e-2c4b-4f66-915b-934fc03eb4bf\") " pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:21:50.615641 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:50.615358 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:21:50.615641 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:50.615431 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs podName:6278283e-2c4b-4f66-915b-934fc03eb4bf nodeName:}" failed. No retries permitted until 2026-07-13 19:21:54.615408988 +0000 UTC m=+9.729316677 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs") pod "network-metrics-daemon-sgfvm" (UID: "6278283e-2c4b-4f66-915b-934fc03eb4bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:21:50.615916 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:50.615896 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 19:21:50.615994 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:50.615962 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret podName:5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4 nodeName:}" failed. No retries permitted until 2026-07-13 19:21:54.615943235 +0000 UTC m=+9.729850946 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret") pod "global-pull-secret-syncer-qgb5b" (UID: "5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4") : object "kube-system"/"original-pull-secret" not registered Jul 13 19:21:50.716183 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:50.716143 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzqnt\" (UniqueName: \"kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt\") pod \"network-check-target-f2lcv\" (UID: \"83623552-aebf-463b-acc9-4cedaa7a7feb\") " pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:21:50.716394 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:50.716352 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 19:21:50.716456 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:50.716374 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 19:21:50.716456 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:50.716419 2556 projected.go:196] Error preparing data for projected volume kube-api-access-rzqnt for pod openshift-network-diagnostics/network-check-target-f2lcv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:21:50.716554 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:50.716475 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt podName:83623552-aebf-463b-acc9-4cedaa7a7feb nodeName:}" failed. No retries permitted until 2026-07-13 19:21:54.716458009 +0000 UTC m=+9.830365699 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rzqnt" (UniqueName: "kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt") pod "network-check-target-f2lcv" (UID: "83623552-aebf-463b-acc9-4cedaa7a7feb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:21:50.845165 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:50.845012 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:50.845165 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:50.845151 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:21:50.845583 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:50.845561 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:21:50.845737 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:50.845697 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:21:50.845828 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:50.845759 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:21:50.846337 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:50.846160 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:21:52.844303 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:52.843641 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:52.844303 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:52.843964 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:21:52.844303 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:52.844077 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:21:52.844303 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:52.844144 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:21:52.844303 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:52.844253 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:21:52.845147 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:52.845106 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:21:54.654149 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:54.654055 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs\") pod \"network-metrics-daemon-sgfvm\" (UID: \"6278283e-2c4b-4f66-915b-934fc03eb4bf\") " pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:21:54.654686 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:54.654183 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret\") pod \"global-pull-secret-syncer-qgb5b\" (UID: \"5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4\") " pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:54.654686 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:54.654234 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:21:54.654686 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:54.654285 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 19:21:54.654686 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:54.654322 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs podName:6278283e-2c4b-4f66-915b-934fc03eb4bf nodeName:}" failed. No retries permitted until 2026-07-13 19:22:02.65429796 +0000 UTC m=+17.768205650 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs") pod "network-metrics-daemon-sgfvm" (UID: "6278283e-2c4b-4f66-915b-934fc03eb4bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:21:54.654686 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:54.654360 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret podName:5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:02.654343857 +0000 UTC m=+17.768251546 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret") pod "global-pull-secret-syncer-qgb5b" (UID: "5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4") : object "kube-system"/"original-pull-secret" not registered Jul 13 19:21:54.755530 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:54.755486 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzqnt\" (UniqueName: \"kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt\") pod \"network-check-target-f2lcv\" (UID: \"83623552-aebf-463b-acc9-4cedaa7a7feb\") " pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:21:54.755728 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:54.755661 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 19:21:54.755728 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:54.755687 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 19:21:54.755728 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:54.755700 2556 projected.go:196] Error preparing data for projected volume kube-api-access-rzqnt for pod openshift-network-diagnostics/network-check-target-f2lcv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:21:54.755960 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:54.755759 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt podName:83623552-aebf-463b-acc9-4cedaa7a7feb nodeName:}" failed. No retries permitted until 2026-07-13 19:22:02.755740999 +0000 UTC m=+17.869648703 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rzqnt" (UniqueName: "kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt") pod "network-check-target-f2lcv" (UID: "83623552-aebf-463b-acc9-4cedaa7a7feb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:21:54.844117 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:54.843892 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:21:54.844117 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:54.844039 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:21:54.844360 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:54.844129 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:21:54.844360 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:54.844244 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:21:54.844360 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:54.844285 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:54.844511 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:54.844359 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:21:56.844640 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:56.844313 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:21:56.845163 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:56.844707 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:21:56.845163 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:56.844422 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:56.845163 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:56.844825 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:21:56.845163 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:56.844395 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:21:56.845163 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:56.844896 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:21:58.843478 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:58.843441 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:21:58.843998 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:58.843582 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:21:58.843998 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:58.843578 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:21:58.843998 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:21:58.843635 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:21:58.843998 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:58.843739 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:21:58.843998 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:21:58.843852 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:22:00.843455 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:00.843413 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:22:00.843958 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:00.843421 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:22:00.843958 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:00.843563 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:22:00.843958 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:00.843626 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:22:00.843958 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:00.843424 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:00.843958 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:00.843710 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:22:02.718894 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:02.718849 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs\") pod \"network-metrics-daemon-sgfvm\" (UID: \"6278283e-2c4b-4f66-915b-934fc03eb4bf\") " pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:22:02.719407 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:02.718913 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret\") pod \"global-pull-secret-syncer-qgb5b\" (UID: \"5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4\") " pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:22:02.719407 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:02.719012 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:22:02.719407 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:02.719063 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 19:22:02.719407 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:02.719085 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs podName:6278283e-2c4b-4f66-915b-934fc03eb4bf nodeName:}" failed. No retries permitted until 2026-07-13 19:22:18.719065384 +0000 UTC m=+33.832973092 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs") pod "network-metrics-daemon-sgfvm" (UID: "6278283e-2c4b-4f66-915b-934fc03eb4bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:22:02.719407 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:02.719113 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret podName:5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:18.719097499 +0000 UTC m=+33.833005186 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret") pod "global-pull-secret-syncer-qgb5b" (UID: "5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4") : object "kube-system"/"original-pull-secret" not registered Jul 13 19:22:02.819580 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:02.819536 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzqnt\" (UniqueName: \"kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt\") pod \"network-check-target-f2lcv\" (UID: \"83623552-aebf-463b-acc9-4cedaa7a7feb\") " pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:02.819820 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:02.819743 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 19:22:02.819820 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:02.819769 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 19:22:02.819820 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:02.819796 2556 projected.go:196] Error preparing data for projected volume kube-api-access-rzqnt for pod openshift-network-diagnostics/network-check-target-f2lcv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:22:02.819974 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:02.819861 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt podName:83623552-aebf-463b-acc9-4cedaa7a7feb nodeName:}" failed. No retries permitted until 2026-07-13 19:22:18.819844976 +0000 UTC m=+33.933752679 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rzqnt" (UniqueName: "kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt") pod "network-check-target-f2lcv" (UID: "83623552-aebf-463b-acc9-4cedaa7a7feb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:22:02.842879 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:02.842835 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:22:02.842879 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:02.842853 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:22:02.843115 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:02.842970 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:22:02.843115 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:02.843026 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:02.843208 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:02.843133 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:22:02.843245 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:02.843209 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:22:04.842926 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:04.842891 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:04.842926 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:04.842913 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:22:04.843399 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:04.842921 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:22:04.843399 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:04.843018 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:22:04.843399 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:04.843100 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:22:04.843399 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:04.843171 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:22:06.844368 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:06.844327 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:22:06.844859 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:06.844376 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:22:06.844859 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:06.844453 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:22:06.844859 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:06.844499 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:22:06.844859 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:06.844541 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:06.844859 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:06.844610 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:22:08.843698 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:08.843568 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:22:08.843698 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:08.843568 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:22:08.844305 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:08.843718 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:22:08.844305 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:08.843568 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:08.844305 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:08.843791 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:22:08.844305 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:08.843861 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:22:09.101750 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:09.101714 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4t2sn" event={"ID":"bb78901a-bfbf-4f49-b2a2-2faf6ed439ab","Type":"ContainerStarted","Data":"4cad29f69e0de441b72e62eca6161d3a0b485e375611ae5c9038f0f8340c4d07"} Jul 13 19:22:09.103367 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:09.103341 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-862gg" event={"ID":"12a6461b-fb8a-4d84-a0dc-707b552a14b6","Type":"ContainerStarted","Data":"4af4e41da03dc108aeb2cd43a2be2794a8459a7455d5b9bbf522b7b8f34dd081"} Jul 13 19:22:09.105334 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:09.105303 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-20.ec2.internal" event={"ID":"e49ce7abc689f72d6258a00fc408109c","Type":"ContainerStarted","Data":"37e565e6432d4179c86d0ab135b73996b806b258f6c8feb32ace5c71ca2ba5a9"} Jul 13 19:22:09.107249 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:09.107225 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" event={"ID":"a72fd793-62c4-49b9-b3cb-06d08aa88ec7","Type":"ContainerStarted","Data":"242fade192657ace1f844e0469ab129f684a105b1bdbcfd643963eede3073a73"} Jul 13 19:22:09.107376 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:09.107302 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" event={"ID":"a72fd793-62c4-49b9-b3cb-06d08aa88ec7","Type":"ContainerStarted","Data":"3d88475036e46895d94fb56907599c82f948683e36250de3cec8d9422aa2a823"} Jul 13 19:22:09.126870 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:09.126823 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-20.ec2.internal" podStartSLOduration=22.126807337 podStartE2EDuration="22.126807337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:09.126673392 +0000 UTC m=+24.240581126" watchObservedRunningTime="2026-07-13 19:22:09.126807337 +0000 UTC m=+24.240715045" Jul 13 19:22:09.145235 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:09.145184 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4t2sn" podStartSLOduration=2.808625829 podStartE2EDuration="23.145166091s" totalImagesPullingTime="20.336540262s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:09.144756057 +0000 UTC m=+24.258663789" watchObservedRunningTime="2026-07-13 19:22:09.145166091 +0000 UTC m=+24.259073800" Jul 13 19:22:09.162337 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:09.162197 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-862gg" podStartSLOduration=2.851632904 podStartE2EDuration="23.162180168s" totalImagesPullingTime="20.310547264s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:09.16148645 +0000 UTC m=+24.275394158" watchObservedRunningTime="2026-07-13 19:22:09.162180168 +0000 UTC m=+24.276087876" Jul 13 19:22:10.111120 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.110868 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" event={"ID":"ea52e435-228a-4124-89ee-5dfa0c6fa47f","Type":"ContainerStarted","Data":"543570f542845cdac019f223bf739cee9904311fa6c14beef4e62c700d466751"} Jul 13 19:22:10.112404 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.112377 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rkbds" event={"ID":"c0a858d7-aa1f-4af2-934a-240c6ed35046","Type":"ContainerStarted","Data":"36e0ec5f88e06cb419cb3f71fda120d4a37867e8d9d8deff7a469cae25d8e337"} Jul 13 19:22:10.113994 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.113962 2556 generic.go:356] "Generic (PLEG): container finished" podID="72f86323-f67b-49e2-b010-df5515861ddd" containerID="43e6cad889c47825c180a71d26bdf88790fe55ba366429caed4a0a604ba6823d" exitCode=0 Jul 13 19:22:10.114154 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.114053 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmdrb" event={"ID":"72f86323-f67b-49e2-b010-df5515861ddd","Type":"ContainerDied","Data":"43e6cad889c47825c180a71d26bdf88790fe55ba366429caed4a0a604ba6823d"} Jul 13 19:22:10.115492 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.115450 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wzrvz" event={"ID":"bd6bcf64-ab9b-4c5f-b23b-07f454be4c24","Type":"ContainerStarted","Data":"47d311eec6f453c7c4fe8c78f960b2782b46d9bdb8b9b4eb838d769e9dc29409"} Jul 13 19:22:10.117024 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.116997 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nlq97" event={"ID":"323bf873-3fca-4fa4-82c9-f2f3be457ed3","Type":"ContainerStarted","Data":"dc8e258962fd516cf8cc9f9136fe95c9527e5f641f774693ffc364ee940a2f4e"} Jul 13 19:22:10.118484 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.118458 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5fj4k" event={"ID":"9b5b0a0b-15eb-4d6d-83cf-117a9009583b","Type":"ContainerStarted","Data":"60b3e56f2fb42baa74fb0b83be52e2bf4e34bbedfdaaa8891aee64c3d9ba69bb"} Jul 13 19:22:10.119944 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.119921 2556 generic.go:356] "Generic (PLEG): container finished" podID="84571915a3a9452655334a0c5a120bd4" containerID="67a1d31e9a093b0f99c349c3968f46e151b09b4d975e96f93519101ed4e0fa98" exitCode=0 Jul 13 19:22:10.120031 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.119997 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal" event={"ID":"84571915a3a9452655334a0c5a120bd4","Type":"ContainerDied","Data":"67a1d31e9a093b0f99c349c3968f46e151b09b4d975e96f93519101ed4e0fa98"} Jul 13 19:22:10.123047 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.123026 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 19:22:10.123357 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.123337 2556 generic.go:356] "Generic (PLEG): container finished" podID="a72fd793-62c4-49b9-b3cb-06d08aa88ec7" containerID="242fade192657ace1f844e0469ab129f684a105b1bdbcfd643963eede3073a73" exitCode=1 Jul 13 19:22:10.123438 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.123420 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" event={"ID":"a72fd793-62c4-49b9-b3cb-06d08aa88ec7","Type":"ContainerDied","Data":"242fade192657ace1f844e0469ab129f684a105b1bdbcfd643963eede3073a73"} Jul 13 19:22:10.123496 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.123484 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" event={"ID":"a72fd793-62c4-49b9-b3cb-06d08aa88ec7","Type":"ContainerStarted","Data":"fec7492c89e11c960cd9a8e947898240a95aae55828881048bb9c1b73225494c"} Jul 13 19:22:10.123536 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.123524 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" event={"ID":"a72fd793-62c4-49b9-b3cb-06d08aa88ec7","Type":"ContainerStarted","Data":"746f769df11ce32f89539fad4135e98572deade86332280216f73b3068ba7019"} Jul 13 19:22:10.123569 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.123558 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" event={"ID":"a72fd793-62c4-49b9-b3cb-06d08aa88ec7","Type":"ContainerStarted","Data":"cd96e93b18e2a88157fb7596defd0478de17b916c983734729d16a43bea2a112"} Jul 13 19:22:10.123611 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.123587 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" event={"ID":"a72fd793-62c4-49b9-b3cb-06d08aa88ec7","Type":"ContainerStarted","Data":"204ee2a4051f4a5e18dc6a10637c40aa58a502e42e07809c6a1bb11e91a5ba23"} Jul 13 19:22:10.127398 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.127361 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rkbds" podStartSLOduration=3.653984468 podStartE2EDuration="24.127349014s" totalImagesPullingTime="20.473364546s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:10.12672396 +0000 UTC m=+25.240631683" watchObservedRunningTime="2026-07-13 19:22:10.127349014 +0000 UTC m=+25.241256767" Jul 13 19:22:10.142854 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.141521 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wzrvz" podStartSLOduration=3.606872478 podStartE2EDuration="24.141503426s" totalImagesPullingTime="20.534630948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:10.140976653 +0000 UTC m=+25.254884383" watchObservedRunningTime="2026-07-13 19:22:10.141503426 +0000 UTC m=+25.255411136" Jul 13 19:22:10.197759 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.197710 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5fj4k" podStartSLOduration=3.657311553 podStartE2EDuration="24.197689026s" totalImagesPullingTime="20.540377473s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:10.196979537 +0000 UTC m=+25.310887290" watchObservedRunningTime="2026-07-13 19:22:10.197689026 +0000 UTC m=+25.311596735" Jul 13 19:22:10.247587 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.247547 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wzrvz" Jul 13 19:22:10.248128 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.248102 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wzrvz" Jul 13 19:22:10.259991 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.259929 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nlq97" podStartSLOduration=3.696742446 podStartE2EDuration="24.259910007s" totalImagesPullingTime="20.563167561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:10.224464118 +0000 UTC m=+25.338371828" watchObservedRunningTime="2026-07-13 19:22:10.259910007 +0000 UTC m=+25.373817716" Jul 13 19:22:10.842971 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.842893 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:22:10.843248 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.842902 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:22:10.843248 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:10.843046 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:22:10.843248 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:10.843056 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:10.843248 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:10.843152 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:22:10.843248 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:10.843238 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:22:11.130901 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:11.130837 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal" event={"ID":"84571915a3a9452655334a0c5a120bd4","Type":"ContainerStarted","Data":"0c804fd6c450426bc8b599022c10691a7de9c3fcd34b02b219ac6e1c1e51f8ea"} Jul 13 19:22:11.131455 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:11.131049 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wzrvz" Jul 13 19:22:11.131678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:11.131658 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wzrvz" Jul 13 19:22:11.147004 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:11.146954 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-20.ec2.internal" podStartSLOduration=24.146930638 podStartE2EDuration="24.146930638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:11.146239903 +0000 UTC m=+26.260147613" watchObservedRunningTime="2026-07-13 19:22:11.146930638 +0000 UTC m=+26.260838346" Jul 13 19:22:11.165390 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:11.165357 2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 19:22:11.853667 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:11.853541 2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T19:22:11.165385302Z","UUID":"8babe93f-8313-4014-b258-32c1d76beeff","Handler":null,"Name":"","Endpoint":""} Jul 13 19:22:11.855715 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:11.855687 2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 19:22:11.855905 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:11.855734 2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 19:22:12.137434 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:12.137357 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 19:22:12.138050 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:12.137766 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" event={"ID":"a72fd793-62c4-49b9-b3cb-06d08aa88ec7","Type":"ContainerStarted","Data":"c7e89aa4213c991f36ae30119ec9bb81f761f46ca4e2d6475a9c57c0822e208b"} Jul 13 19:22:12.139970 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:12.139931 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" event={"ID":"ea52e435-228a-4124-89ee-5dfa0c6fa47f","Type":"ContainerStarted","Data":"80f5340d5e688a916652c654ac0059e698d71247a4730c86be7980140de18ae9"} Jul 13 19:22:12.843464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:12.843378 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:12.843464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:12.843378 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:22:12.843677 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:12.843509 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:22:12.843677 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:12.843541 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:22:12.843677 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:12.843378 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:22:12.843677 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:12.843633 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:22:13.147510 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:13.147424 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" event={"ID":"ea52e435-228a-4124-89ee-5dfa0c6fa47f","Type":"ContainerStarted","Data":"0249bed5c078bbc81c619651e38a138e2dd56ac5707eab09f20e54b786c068a7"} Jul 13 19:22:13.176882 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:13.176484 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jckhg" podStartSLOduration=3.859397917 podStartE2EDuration="27.176444446s" totalImagesPullingTime="23.317046529s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:13.174318254 +0000 UTC m=+28.288225964" watchObservedRunningTime="2026-07-13 19:22:13.176444446 +0000 UTC m=+28.290352157" Jul 13 19:22:14.843354 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:14.843258 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:22:14.844129 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:14.843270 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:14.844129 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:14.843377 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:22:14.844129 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:14.843270 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:22:14.844129 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:14.843447 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:22:14.844129 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:14.843573 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:22:15.154719 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:15.154631 2556 generic.go:356] "Generic (PLEG): container finished" podID="72f86323-f67b-49e2-b010-df5515861ddd" containerID="cde011eaf004129224a915e5ec1d8f1d6172de308d14681d8121f38df8c4132a" exitCode=0 Jul 13 19:22:15.154719 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:15.154711 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmdrb" event={"ID":"72f86323-f67b-49e2-b010-df5515861ddd","Type":"ContainerDied","Data":"cde011eaf004129224a915e5ec1d8f1d6172de308d14681d8121f38df8c4132a"} Jul 13 19:22:15.158445 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:15.158279 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 19:22:15.158804 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:15.158759 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" event={"ID":"a72fd793-62c4-49b9-b3cb-06d08aa88ec7","Type":"ContainerStarted","Data":"5bc54c1627989e2dc7ba08603dceeedf5d539cdfa44bb4b348e25974b5acbcac"} Jul 13 19:22:15.159136 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:15.159111 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:22:15.159261 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:15.159185 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:22:15.159356 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:15.159340 2556 scope.go:117] "RemoveContainer" containerID="242fade192657ace1f844e0469ab129f684a105b1bdbcfd643963eede3073a73" Jul 13 19:22:15.175970 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:15.175947 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:22:16.166922 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:16.166733 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 19:22:16.167627 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:16.167581 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" event={"ID":"a72fd793-62c4-49b9-b3cb-06d08aa88ec7","Type":"ContainerStarted","Data":"b423a229bd9c265af42feb3da41f5634915a32508f45987fa41dd94db44e482c"} Jul 13 19:22:16.167871 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:16.167852 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:22:16.188858 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:16.188830 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:22:16.201562 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:16.201492 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" podStartSLOduration=9.380573868 podStartE2EDuration="30.201469085s" totalImagesPullingTime="20.820895217s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:16.200752441 +0000 UTC m=+31.314660188" watchObservedRunningTime="2026-07-13 19:22:16.201469085 +0000 UTC m=+31.315376794" Jul 13 19:22:16.330601 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:16.330462 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qgb5b"] Jul 13 19:22:16.330601 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:16.330599 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:22:16.330910 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:16.330712 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:22:16.331541 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:16.331509 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f2lcv"] Jul 13 19:22:16.331714 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:16.331624 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:16.331807 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:16.331716 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:22:16.332189 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:16.332159 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sgfvm"] Jul 13 19:22:16.332329 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:16.332274 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:22:16.332390 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:16.332369 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:22:17.171723 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:17.171690 2556 generic.go:356] "Generic (PLEG): container finished" podID="72f86323-f67b-49e2-b010-df5515861ddd" containerID="860455e226c7006e3648b68822e469c35d7a3d1c1ff5c04982ac4a94e876fa46" exitCode=0 Jul 13 19:22:17.172151 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:17.171799 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmdrb" event={"ID":"72f86323-f67b-49e2-b010-df5515861ddd","Type":"ContainerDied","Data":"860455e226c7006e3648b68822e469c35d7a3d1c1ff5c04982ac4a94e876fa46"} Jul 13 19:22:17.843342 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:17.843305 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:17.843342 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:17.843344 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:22:17.843603 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:17.843343 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:22:17.843603 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:17.843414 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:22:17.843603 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:17.843510 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:22:17.843603 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:17.843562 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:22:18.732699 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:18.732460 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret\") pod \"global-pull-secret-syncer-qgb5b\" (UID: \"5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4\") " pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:22:18.733102 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:18.732615 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 19:22:18.733102 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:18.732851 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:22:18.733102 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:18.732858 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret podName:5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:50.732833217 +0000 UTC m=+65.846740922 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret") pod "global-pull-secret-syncer-qgb5b" (UID: "5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4") : object "kube-system"/"original-pull-secret" not registered Jul 13 19:22:18.733102 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:18.732761 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs\") pod \"network-metrics-daemon-sgfvm\" (UID: \"6278283e-2c4b-4f66-915b-934fc03eb4bf\") " pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:22:18.733102 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:18.732885 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs podName:6278283e-2c4b-4f66-915b-934fc03eb4bf nodeName:}" failed. No retries permitted until 2026-07-13 19:22:50.732875886 +0000 UTC m=+65.846783574 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs") pod "network-metrics-daemon-sgfvm" (UID: "6278283e-2c4b-4f66-915b-934fc03eb4bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 19:22:18.833762 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:18.833706 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzqnt\" (UniqueName: \"kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt\") pod \"network-check-target-f2lcv\" (UID: \"83623552-aebf-463b-acc9-4cedaa7a7feb\") " pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:18.833953 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:18.833904 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 19:22:18.833953 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:18.833928 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 19:22:18.833953 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:18.833939 2556 projected.go:196] Error preparing data for projected volume kube-api-access-rzqnt for pod openshift-network-diagnostics/network-check-target-f2lcv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:22:18.834056 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:18.834003 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt podName:83623552-aebf-463b-acc9-4cedaa7a7feb nodeName:}" failed. No retries permitted until 2026-07-13 19:22:50.833984086 +0000 UTC m=+65.947891772 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rzqnt" (UniqueName: "kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt") pod "network-check-target-f2lcv" (UID: "83623552-aebf-463b-acc9-4cedaa7a7feb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 19:22:19.177711 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:19.177678 2556 generic.go:356] "Generic (PLEG): container finished" podID="72f86323-f67b-49e2-b010-df5515861ddd" containerID="f49f759ff6a11580dd93841ac1694e31e78925f9d4d4778e8ad4d9065aea6092" exitCode=0 Jul 13 19:22:19.177711 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:19.177718 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmdrb" event={"ID":"72f86323-f67b-49e2-b010-df5515861ddd","Type":"ContainerDied","Data":"f49f759ff6a11580dd93841ac1694e31e78925f9d4d4778e8ad4d9065aea6092"} Jul 13 19:22:19.843012 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:19.842973 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:22:19.843467 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:19.842973 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:22:19.843467 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:19.843093 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:22:19.843467 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:19.842973 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:19.843467 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:19.843160 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:22:19.843467 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:19.843232 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:22:21.842867 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:21.842767 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:21.842867 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:21.842831 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:22:21.842867 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:21.842834 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:22:21.843444 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:21.842919 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2lcv" podUID="83623552-aebf-463b-acc9-4cedaa7a7feb" Jul 13 19:22:21.843444 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:21.843037 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgfvm" podUID="6278283e-2c4b-4f66-915b-934fc03eb4bf" Jul 13 19:22:21.843444 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:21.843115 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgb5b" podUID="5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4" Jul 13 19:22:22.734492 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.734458 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-20.ec2.internal" event="NodeReady" Jul 13 19:22:22.734702 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.734640 2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 19:22:22.802625 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.802567 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fplcj"] Jul 13 19:22:22.805192 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.805164 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-k24np"] Jul 13 19:22:22.805346 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.805309 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fplcj" Jul 13 19:22:22.807130 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.807101 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k24np" Jul 13 19:22:22.807661 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.807632 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 19:22:22.807812 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.807732 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bvq9g\"" Jul 13 19:22:22.807903 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.807831 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 19:22:22.807903 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.807889 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 19:22:22.808846 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.808826 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mf9hk\"" Jul 13 19:22:22.808996 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.808974 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 19:22:22.809200 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.809186 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 19:22:22.815444 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.815407 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fplcj"] Jul 13 19:22:22.821243 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.821132 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-k24np"] Jul 13 19:22:22.961988 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.961950 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23b8d70a-3533-428f-9b09-ad9c365fb03d-cert\") pod \"ingress-canary-fplcj\" (UID: \"23b8d70a-3533-428f-9b09-ad9c365fb03d\") " pod="openshift-ingress-canary/ingress-canary-fplcj" Jul 13 19:22:22.961988 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.961995 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfxzm\" (UniqueName: \"kubernetes.io/projected/ee1b48b4-0511-4f05-8294-434364af55c1-kube-api-access-zfxzm\") pod \"dns-default-k24np\" (UID: \"ee1b48b4-0511-4f05-8294-434364af55c1\") " pod="openshift-dns/dns-default-k24np" Jul 13 19:22:22.962644 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.962059 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls\") pod \"dns-default-k24np\" (UID: \"ee1b48b4-0511-4f05-8294-434364af55c1\") " pod="openshift-dns/dns-default-k24np" Jul 13 19:22:22.962644 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.962114 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdthc\" (UniqueName: \"kubernetes.io/projected/23b8d70a-3533-428f-9b09-ad9c365fb03d-kube-api-access-hdthc\") pod \"ingress-canary-fplcj\" (UID: \"23b8d70a-3533-428f-9b09-ad9c365fb03d\") " pod="openshift-ingress-canary/ingress-canary-fplcj" Jul 13 19:22:22.962644 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.962141 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ee1b48b4-0511-4f05-8294-434364af55c1-config-volume\") pod \"dns-default-k24np\" (UID: \"ee1b48b4-0511-4f05-8294-434364af55c1\") " pod="openshift-dns/dns-default-k24np" Jul 13 19:22:22.962644 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:22.962169 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee1b48b4-0511-4f05-8294-434364af55c1-tmp-dir\") pod \"dns-default-k24np\" (UID: \"ee1b48b4-0511-4f05-8294-434364af55c1\") " pod="openshift-dns/dns-default-k24np" Jul 13 19:22:23.063483 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.063366 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23b8d70a-3533-428f-9b09-ad9c365fb03d-cert\") pod \"ingress-canary-fplcj\" (UID: \"23b8d70a-3533-428f-9b09-ad9c365fb03d\") " pod="openshift-ingress-canary/ingress-canary-fplcj" Jul 13 19:22:23.063483 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.063428 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfxzm\" (UniqueName: \"kubernetes.io/projected/ee1b48b4-0511-4f05-8294-434364af55c1-kube-api-access-zfxzm\") pod \"dns-default-k24np\" (UID: \"ee1b48b4-0511-4f05-8294-434364af55c1\") " pod="openshift-dns/dns-default-k24np" Jul 13 19:22:23.063483 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.063466 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls\") pod \"dns-default-k24np\" (UID: \"ee1b48b4-0511-4f05-8294-434364af55c1\") " pod="openshift-dns/dns-default-k24np" Jul 13 19:22:23.063764 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.063501 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdthc\" (UniqueName: \"kubernetes.io/projected/23b8d70a-3533-428f-9b09-ad9c365fb03d-kube-api-access-hdthc\") pod \"ingress-canary-fplcj\" (UID: \"23b8d70a-3533-428f-9b09-ad9c365fb03d\") " pod="openshift-ingress-canary/ingress-canary-fplcj" Jul 13 19:22:23.063764 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.063531 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ee1b48b4-0511-4f05-8294-434364af55c1-config-volume\") pod \"dns-default-k24np\" (UID: \"ee1b48b4-0511-4f05-8294-434364af55c1\") " pod="openshift-dns/dns-default-k24np" Jul 13 19:22:23.063764 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.063558 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee1b48b4-0511-4f05-8294-434364af55c1-tmp-dir\") pod \"dns-default-k24np\" (UID: \"ee1b48b4-0511-4f05-8294-434364af55c1\") " pod="openshift-dns/dns-default-k24np" Jul 13 19:22:23.063994 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:23.063856 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 19:22:23.064258 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:23.064228 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls podName:ee1b48b4-0511-4f05-8294-434364af55c1 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:23.564068961 +0000 UTC m=+38.677976651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls") pod "dns-default-k24np" (UID: "ee1b48b4-0511-4f05-8294-434364af55c1") : secret "dns-default-metrics-tls" not found Jul 13 19:22:23.065207 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.064614 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ee1b48b4-0511-4f05-8294-434364af55c1-config-volume\") pod \"dns-default-k24np\" (UID: \"ee1b48b4-0511-4f05-8294-434364af55c1\") " pod="openshift-dns/dns-default-k24np" Jul 13 19:22:23.065207 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.064764 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee1b48b4-0511-4f05-8294-434364af55c1-tmp-dir\") pod \"dns-default-k24np\" (UID: \"ee1b48b4-0511-4f05-8294-434364af55c1\") " pod="openshift-dns/dns-default-k24np" Jul 13 19:22:23.070810 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.070703 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23b8d70a-3533-428f-9b09-ad9c365fb03d-cert\") pod \"ingress-canary-fplcj\" (UID: \"23b8d70a-3533-428f-9b09-ad9c365fb03d\") " pod="openshift-ingress-canary/ingress-canary-fplcj" Jul 13 19:22:23.073416 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.073383 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfxzm\" (UniqueName: \"kubernetes.io/projected/ee1b48b4-0511-4f05-8294-434364af55c1-kube-api-access-zfxzm\") pod \"dns-default-k24np\" (UID: \"ee1b48b4-0511-4f05-8294-434364af55c1\") " pod="openshift-dns/dns-default-k24np" Jul 13 19:22:23.073562 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.073383 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdthc\" (UniqueName: \"kubernetes.io/projected/23b8d70a-3533-428f-9b09-ad9c365fb03d-kube-api-access-hdthc\") pod \"ingress-canary-fplcj\" (UID: \"23b8d70a-3533-428f-9b09-ad9c365fb03d\") " pod="openshift-ingress-canary/ingress-canary-fplcj" Jul 13 19:22:23.118309 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.118267 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fplcj" Jul 13 19:22:23.567614 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.567577 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls\") pod \"dns-default-k24np\" (UID: \"ee1b48b4-0511-4f05-8294-434364af55c1\") " pod="openshift-dns/dns-default-k24np" Jul 13 19:22:23.567819 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:23.567730 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 19:22:23.567866 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:23.567826 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls podName:ee1b48b4-0511-4f05-8294-434364af55c1 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:24.567804788 +0000 UTC m=+39.681712514 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls") pod "dns-default-k24np" (UID: "ee1b48b4-0511-4f05-8294-434364af55c1") : secret "dns-default-metrics-tls" not found Jul 13 19:22:23.844111 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.843458 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:22:23.844111 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.843478 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:23.844111 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.843499 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:22:23.846698 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.846658 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 19:22:23.847021 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.847004 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 19:22:23.847709 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.847689 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 19:22:23.847709 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.847697 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ztp4q\"" Jul 13 19:22:23.847897 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.847722 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fp668\"" Jul 13 19:22:23.847956 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.847945 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 19:22:23.890450 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:23.890426 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fplcj"] Jul 13 19:22:23.892981 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:22:23.892950 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23b8d70a_3533_428f_9b09_ad9c365fb03d.slice/crio-01609f2e0a710a2065b163c439f21f4217c1ff679fd7b7ecf9d6abc5fafdb6ac WatchSource:0}: Error finding container 01609f2e0a710a2065b163c439f21f4217c1ff679fd7b7ecf9d6abc5fafdb6ac: Status 404 returned error can't find the container with id 01609f2e0a710a2065b163c439f21f4217c1ff679fd7b7ecf9d6abc5fafdb6ac Jul 13 19:22:24.195172 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:24.195134 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fplcj" event={"ID":"23b8d70a-3533-428f-9b09-ad9c365fb03d","Type":"ContainerStarted","Data":"01609f2e0a710a2065b163c439f21f4217c1ff679fd7b7ecf9d6abc5fafdb6ac"} Jul 13 19:22:24.198045 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:24.198016 2556 generic.go:356] "Generic (PLEG): container finished" podID="72f86323-f67b-49e2-b010-df5515861ddd" containerID="a207089f046fce7aac9eb545ab41a9472de18405a3e89a75a7cbbd8ed4fd7c07" exitCode=0 Jul 13 19:22:24.198192 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:24.198053 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmdrb" event={"ID":"72f86323-f67b-49e2-b010-df5515861ddd","Type":"ContainerDied","Data":"a207089f046fce7aac9eb545ab41a9472de18405a3e89a75a7cbbd8ed4fd7c07"} Jul 13 19:22:24.575469 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:24.575429 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls\") pod \"dns-default-k24np\" (UID: \"ee1b48b4-0511-4f05-8294-434364af55c1\") " pod="openshift-dns/dns-default-k24np" Jul 13 19:22:24.575629 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:24.575584 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 19:22:24.575670 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:24.575656 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls podName:ee1b48b4-0511-4f05-8294-434364af55c1 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:26.575635971 +0000 UTC m=+41.689543658 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls") pod "dns-default-k24np" (UID: "ee1b48b4-0511-4f05-8294-434364af55c1") : secret "dns-default-metrics-tls" not found Jul 13 19:22:25.204250 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:25.204209 2556 generic.go:356] "Generic (PLEG): container finished" podID="72f86323-f67b-49e2-b010-df5515861ddd" containerID="eb52e7b90cfe71f8538bb6b7d569200d5338d3b7802254fcb4940f9e42fad431" exitCode=0 Jul 13 19:22:25.204871 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:25.204248 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmdrb" event={"ID":"72f86323-f67b-49e2-b010-df5515861ddd","Type":"ContainerDied","Data":"eb52e7b90cfe71f8538bb6b7d569200d5338d3b7802254fcb4940f9e42fad431"} Jul 13 19:22:26.211295 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:26.211181 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmdrb" event={"ID":"72f86323-f67b-49e2-b010-df5515861ddd","Type":"ContainerStarted","Data":"30450f359621b5550f3dcd1fa84653de000854f7798c18eee03594fd165de844"} Jul 13 19:22:26.234579 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:26.234457 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lmdrb" podStartSLOduration=8.287968405 podStartE2EDuration="40.23443595s" totalImagesPullingTime="31.946467545s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:26.234125863 +0000 UTC m=+41.348033683" watchObservedRunningTime="2026-07-13 19:22:26.23443595 +0000 UTC m=+41.348343660" Jul 13 19:22:26.592807 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:26.592698 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls\") pod \"dns-default-k24np\" (UID: \"ee1b48b4-0511-4f05-8294-434364af55c1\") " pod="openshift-dns/dns-default-k24np" Jul 13 19:22:26.592986 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:26.592876 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 19:22:26.592986 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:26.592951 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls podName:ee1b48b4-0511-4f05-8294-434364af55c1 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:30.592931662 +0000 UTC m=+45.706839348 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls") pod "dns-default-k24np" (UID: "ee1b48b4-0511-4f05-8294-434364af55c1") : secret "dns-default-metrics-tls" not found Jul 13 19:22:27.215362 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:27.215257 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fplcj" event={"ID":"23b8d70a-3533-428f-9b09-ad9c365fb03d","Type":"ContainerStarted","Data":"076b3e1249c710491f84eb1a08e39d25a68bcd1a71a99303236ca043849ff05c"} Jul 13 19:22:27.229969 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:27.229916 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fplcj" podStartSLOduration=2.284904488 podStartE2EDuration="5.229896369s" totalImagesPullingTime="2.944991881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:22:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:27.229152884 +0000 UTC m=+42.343060622" watchObservedRunningTime="2026-07-13 19:22:27.229896369 +0000 UTC m=+42.343804078" Jul 13 19:22:28.787899 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:28.787865 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-r4flg"] Jul 13 19:22:28.810824 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:28.810787 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r4flg"] Jul 13 19:22:28.811002 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:28.810919 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:28.812715 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:28.812691 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 19:22:28.813007 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:28.812990 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 19:22:28.813090 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:28.813036 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 19:22:28.813272 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:28.813255 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 19:22:28.813339 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:28.813254 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dkwp7\"" Jul 13 19:22:28.910072 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:28.910031 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/244c17d0-fb0f-4a75-a802-e99dd9dc3556-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:28.910266 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:28.910140 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/244c17d0-fb0f-4a75-a802-e99dd9dc3556-data-volume\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:28.910266 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:28.910174 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/244c17d0-fb0f-4a75-a802-e99dd9dc3556-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:28.910266 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:28.910196 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/244c17d0-fb0f-4a75-a802-e99dd9dc3556-crio-socket\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:28.910266 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:28.910213 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98pdn\" (UniqueName: \"kubernetes.io/projected/244c17d0-fb0f-4a75-a802-e99dd9dc3556-kube-api-access-98pdn\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:29.010821 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:29.010761 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/244c17d0-fb0f-4a75-a802-e99dd9dc3556-data-volume\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:29.010821 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:29.010824 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/244c17d0-fb0f-4a75-a802-e99dd9dc3556-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:29.011041 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:29.010891 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/244c17d0-fb0f-4a75-a802-e99dd9dc3556-crio-socket\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:29.011041 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:29.010896 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 19:22:29.011041 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:29.010914 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98pdn\" (UniqueName: \"kubernetes.io/projected/244c17d0-fb0f-4a75-a802-e99dd9dc3556-kube-api-access-98pdn\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:29.011041 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:29.010967 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/244c17d0-fb0f-4a75-a802-e99dd9dc3556-kube-rbac-proxy-cm podName:244c17d0-fb0f-4a75-a802-e99dd9dc3556 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:29.510948265 +0000 UTC m=+44.624855952 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/244c17d0-fb0f-4a75-a802-e99dd9dc3556-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r4flg" (UID: "244c17d0-fb0f-4a75-a802-e99dd9dc3556") : configmap "kube-rbac-proxy" not found Jul 13 19:22:29.011041 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:29.011034 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/244c17d0-fb0f-4a75-a802-e99dd9dc3556-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:29.011259 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:29.011139 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/244c17d0-fb0f-4a75-a802-e99dd9dc3556-crio-socket\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:29.011259 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:29.011175 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/244c17d0-fb0f-4a75-a802-e99dd9dc3556-data-volume\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:29.011259 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:29.011190 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 19:22:29.011259 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:29.011241 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244c17d0-fb0f-4a75-a802-e99dd9dc3556-insights-runtime-extractor-tls podName:244c17d0-fb0f-4a75-a802-e99dd9dc3556 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:29.511228198 +0000 UTC m=+44.625135889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/244c17d0-fb0f-4a75-a802-e99dd9dc3556-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r4flg" (UID: "244c17d0-fb0f-4a75-a802-e99dd9dc3556") : secret "insights-runtime-extractor-tls" not found Jul 13 19:22:29.021009 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:29.020975 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98pdn\" (UniqueName: \"kubernetes.io/projected/244c17d0-fb0f-4a75-a802-e99dd9dc3556-kube-api-access-98pdn\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:29.514097 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:29.514050 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/244c17d0-fb0f-4a75-a802-e99dd9dc3556-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:29.514281 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:29.514126 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/244c17d0-fb0f-4a75-a802-e99dd9dc3556-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:29.514281 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:29.514207 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 19:22:29.514281 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:29.514272 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/244c17d0-fb0f-4a75-a802-e99dd9dc3556-kube-rbac-proxy-cm podName:244c17d0-fb0f-4a75-a802-e99dd9dc3556 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:30.514258273 +0000 UTC m=+45.628165960 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/244c17d0-fb0f-4a75-a802-e99dd9dc3556-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r4flg" (UID: "244c17d0-fb0f-4a75-a802-e99dd9dc3556") : configmap "kube-rbac-proxy" not found Jul 13 19:22:29.514391 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:29.514205 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 19:22:29.514391 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:29.514344 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244c17d0-fb0f-4a75-a802-e99dd9dc3556-insights-runtime-extractor-tls podName:244c17d0-fb0f-4a75-a802-e99dd9dc3556 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:30.514330269 +0000 UTC m=+45.628237956 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/244c17d0-fb0f-4a75-a802-e99dd9dc3556-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r4flg" (UID: "244c17d0-fb0f-4a75-a802-e99dd9dc3556") : secret "insights-runtime-extractor-tls" not found Jul 13 19:22:30.522546 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:30.522495 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/244c17d0-fb0f-4a75-a802-e99dd9dc3556-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:30.523003 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:30.522581 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/244c17d0-fb0f-4a75-a802-e99dd9dc3556-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:30.523003 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:30.522620 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 19:22:30.523003 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:30.522691 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/244c17d0-fb0f-4a75-a802-e99dd9dc3556-kube-rbac-proxy-cm podName:244c17d0-fb0f-4a75-a802-e99dd9dc3556 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:32.522675823 +0000 UTC m=+47.636583510 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/244c17d0-fb0f-4a75-a802-e99dd9dc3556-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r4flg" (UID: "244c17d0-fb0f-4a75-a802-e99dd9dc3556") : configmap "kube-rbac-proxy" not found Jul 13 19:22:30.523003 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:30.522706 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 19:22:30.523003 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:30.522745 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244c17d0-fb0f-4a75-a802-e99dd9dc3556-insights-runtime-extractor-tls podName:244c17d0-fb0f-4a75-a802-e99dd9dc3556 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:32.522734092 +0000 UTC m=+47.636641780 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/244c17d0-fb0f-4a75-a802-e99dd9dc3556-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r4flg" (UID: "244c17d0-fb0f-4a75-a802-e99dd9dc3556") : secret "insights-runtime-extractor-tls" not found Jul 13 19:22:30.623548 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:30.623501 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls\") pod \"dns-default-k24np\" (UID: \"ee1b48b4-0511-4f05-8294-434364af55c1\") " pod="openshift-dns/dns-default-k24np" Jul 13 19:22:30.623733 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:30.623641 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 19:22:30.623733 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:30.623716 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls podName:ee1b48b4-0511-4f05-8294-434364af55c1 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:38.62369875 +0000 UTC m=+53.737606437 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls") pod "dns-default-k24np" (UID: "ee1b48b4-0511-4f05-8294-434364af55c1") : secret "dns-default-metrics-tls" not found Jul 13 19:22:30.674178 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:30.674152 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5fj4k_9b5b0a0b-15eb-4d6d-83cf-117a9009583b/dns-node-resolver/0.log" Jul 13 19:22:30.763134 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:30.763100 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nrc2n"] Jul 13 19:22:30.794175 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:30.794142 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nrc2n"] Jul 13 19:22:30.794378 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:30.794285 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nrc2n" Jul 13 19:22:30.796576 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:30.796543 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 13 19:22:30.797082 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:30.797068 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-z6zkj\"" Jul 13 19:22:30.797158 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:30.797081 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 13 19:22:30.926085 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:30.926044 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgxrx\" (UniqueName: \"kubernetes.io/projected/6a7d1437-bb2d-4b24-857c-7dde4b406770-kube-api-access-mgxrx\") pod \"migrator-7f59b54f6c-nrc2n\" (UID: \"6a7d1437-bb2d-4b24-857c-7dde4b406770\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nrc2n" Jul 13 19:22:31.027404 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:31.027364 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgxrx\" (UniqueName: \"kubernetes.io/projected/6a7d1437-bb2d-4b24-857c-7dde4b406770-kube-api-access-mgxrx\") pod \"migrator-7f59b54f6c-nrc2n\" (UID: \"6a7d1437-bb2d-4b24-857c-7dde4b406770\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nrc2n" Jul 13 19:22:31.036586 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:31.036540 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgxrx\" (UniqueName: \"kubernetes.io/projected/6a7d1437-bb2d-4b24-857c-7dde4b406770-kube-api-access-mgxrx\") pod \"migrator-7f59b54f6c-nrc2n\" (UID: \"6a7d1437-bb2d-4b24-857c-7dde4b406770\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nrc2n" Jul 13 19:22:31.104040 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:31.103937 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nrc2n" Jul 13 19:22:31.252490 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:31.252457 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nrc2n"] Jul 13 19:22:31.256286 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:22:31.256251 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a7d1437_bb2d_4b24_857c_7dde4b406770.slice/crio-9d9db312961c9029502d0389202f4207bfb0ec72c7b78c066ad0759d1a920f32 WatchSource:0}: Error finding container 9d9db312961c9029502d0389202f4207bfb0ec72c7b78c066ad0759d1a920f32: Status 404 returned error can't find the container with id 9d9db312961c9029502d0389202f4207bfb0ec72c7b78c066ad0759d1a920f32 Jul 13 19:22:31.957554 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:31.957523 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-7m2hd"] Jul 13 19:22:31.976429 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:31.976388 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-7m2hd"] Jul 13 19:22:31.976625 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:31.976538 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-7m2hd" Jul 13 19:22:31.979028 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:31.979004 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 13 19:22:31.979177 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:31.979005 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 13 19:22:31.979177 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:31.979051 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 13 19:22:31.979405 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:31.979386 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 13 19:22:31.979495 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:31.979426 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-vh2t8\"" Jul 13 19:22:32.073589 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:32.073555 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rkbds_c0a858d7-aa1f-4af2-934a-240c6ed35046/node-ca/0.log" Jul 13 19:22:32.135189 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:32.135137 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1d5a2de8-d8a3-4f9d-a722-7b514344e080-signing-key\") pod \"service-ca-79f68b9f4f-7m2hd\" (UID: \"1d5a2de8-d8a3-4f9d-a722-7b514344e080\") " pod="openshift-service-ca/service-ca-79f68b9f4f-7m2hd" Jul 13 19:22:32.135189 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:32.135180 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1d5a2de8-d8a3-4f9d-a722-7b514344e080-signing-cabundle\") pod \"service-ca-79f68b9f4f-7m2hd\" (UID: \"1d5a2de8-d8a3-4f9d-a722-7b514344e080\") " pod="openshift-service-ca/service-ca-79f68b9f4f-7m2hd" Jul 13 19:22:32.135503 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:32.135299 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9z9x\" (UniqueName: \"kubernetes.io/projected/1d5a2de8-d8a3-4f9d-a722-7b514344e080-kube-api-access-f9z9x\") pod \"service-ca-79f68b9f4f-7m2hd\" (UID: \"1d5a2de8-d8a3-4f9d-a722-7b514344e080\") " pod="openshift-service-ca/service-ca-79f68b9f4f-7m2hd" Jul 13 19:22:32.228239 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:32.228146 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nrc2n" event={"ID":"6a7d1437-bb2d-4b24-857c-7dde4b406770","Type":"ContainerStarted","Data":"9d9db312961c9029502d0389202f4207bfb0ec72c7b78c066ad0759d1a920f32"} Jul 13 19:22:32.236533 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:32.236495 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9z9x\" (UniqueName: \"kubernetes.io/projected/1d5a2de8-d8a3-4f9d-a722-7b514344e080-kube-api-access-f9z9x\") pod \"service-ca-79f68b9f4f-7m2hd\" (UID: \"1d5a2de8-d8a3-4f9d-a722-7b514344e080\") " pod="openshift-service-ca/service-ca-79f68b9f4f-7m2hd" Jul 13 19:22:32.236683 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:32.236607 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1d5a2de8-d8a3-4f9d-a722-7b514344e080-signing-key\") pod \"service-ca-79f68b9f4f-7m2hd\" (UID: \"1d5a2de8-d8a3-4f9d-a722-7b514344e080\") " pod="openshift-service-ca/service-ca-79f68b9f4f-7m2hd" Jul 13 19:22:32.236683 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:32.236627 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1d5a2de8-d8a3-4f9d-a722-7b514344e080-signing-cabundle\") pod \"service-ca-79f68b9f4f-7m2hd\" (UID: \"1d5a2de8-d8a3-4f9d-a722-7b514344e080\") " pod="openshift-service-ca/service-ca-79f68b9f4f-7m2hd" Jul 13 19:22:32.237434 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:32.237414 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1d5a2de8-d8a3-4f9d-a722-7b514344e080-signing-cabundle\") pod \"service-ca-79f68b9f4f-7m2hd\" (UID: \"1d5a2de8-d8a3-4f9d-a722-7b514344e080\") " pod="openshift-service-ca/service-ca-79f68b9f4f-7m2hd" Jul 13 19:22:32.239139 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:32.239121 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1d5a2de8-d8a3-4f9d-a722-7b514344e080-signing-key\") pod \"service-ca-79f68b9f4f-7m2hd\" (UID: \"1d5a2de8-d8a3-4f9d-a722-7b514344e080\") " pod="openshift-service-ca/service-ca-79f68b9f4f-7m2hd" Jul 13 19:22:32.244672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:32.244647 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9z9x\" (UniqueName: \"kubernetes.io/projected/1d5a2de8-d8a3-4f9d-a722-7b514344e080-kube-api-access-f9z9x\") pod \"service-ca-79f68b9f4f-7m2hd\" (UID: \"1d5a2de8-d8a3-4f9d-a722-7b514344e080\") " pod="openshift-service-ca/service-ca-79f68b9f4f-7m2hd" Jul 13 19:22:32.286138 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:32.286096 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-7m2hd" Jul 13 19:22:32.456885 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:32.456842 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-7m2hd"] Jul 13 19:22:32.461265 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:22:32.461233 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d5a2de8_d8a3_4f9d_a722_7b514344e080.slice/crio-383a6e1364f439d664b0ef2a79213fefea0559a916631f9662c707edb78d47c1 WatchSource:0}: Error finding container 383a6e1364f439d664b0ef2a79213fefea0559a916631f9662c707edb78d47c1: Status 404 returned error can't find the container with id 383a6e1364f439d664b0ef2a79213fefea0559a916631f9662c707edb78d47c1 Jul 13 19:22:32.539206 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:32.539172 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/244c17d0-fb0f-4a75-a802-e99dd9dc3556-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:32.539374 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:32.539250 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/244c17d0-fb0f-4a75-a802-e99dd9dc3556-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:32.539374 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:32.539321 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 19:22:32.539449 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:32.539372 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 19:22:32.539449 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:32.539398 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/244c17d0-fb0f-4a75-a802-e99dd9dc3556-kube-rbac-proxy-cm podName:244c17d0-fb0f-4a75-a802-e99dd9dc3556 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:36.53937812 +0000 UTC m=+51.653285832 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/244c17d0-fb0f-4a75-a802-e99dd9dc3556-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r4flg" (UID: "244c17d0-fb0f-4a75-a802-e99dd9dc3556") : configmap "kube-rbac-proxy" not found Jul 13 19:22:32.539449 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:32.539416 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244c17d0-fb0f-4a75-a802-e99dd9dc3556-insights-runtime-extractor-tls podName:244c17d0-fb0f-4a75-a802-e99dd9dc3556 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:36.539408488 +0000 UTC m=+51.653316175 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/244c17d0-fb0f-4a75-a802-e99dd9dc3556-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r4flg" (UID: "244c17d0-fb0f-4a75-a802-e99dd9dc3556") : secret "insights-runtime-extractor-tls" not found Jul 13 19:22:32.678279 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:32.675518 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fplcj_23b8d70a-3533-428f-9b09-ad9c365fb03d/serve-healthcheck-canary/0.log" Jul 13 19:22:33.231664 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:33.231617 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-7m2hd" event={"ID":"1d5a2de8-d8a3-4f9d-a722-7b514344e080","Type":"ContainerStarted","Data":"383a6e1364f439d664b0ef2a79213fefea0559a916631f9662c707edb78d47c1"} Jul 13 19:22:34.236745 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:34.236698 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nrc2n" event={"ID":"6a7d1437-bb2d-4b24-857c-7dde4b406770","Type":"ContainerStarted","Data":"37ae7060ba83b8b1481459d87a74c8061ae259a7cdb1b625f2db426db9d4a0db"} Jul 13 19:22:34.237254 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:34.236790 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nrc2n" event={"ID":"6a7d1437-bb2d-4b24-857c-7dde4b406770","Type":"ContainerStarted","Data":"da66887377c9db63184846fec3df38190d116236ae00bdb93aec5fbe1d9cd8d9"} Jul 13 19:22:34.253718 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:34.253646 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nrc2n" podStartSLOduration=2.113253463 podStartE2EDuration="4.25362331s" totalImagesPullingTime="2.140369847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:22:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:34.252249924 +0000 UTC m=+49.366157643" watchObservedRunningTime="2026-07-13 19:22:34.25362331 +0000 UTC m=+49.367531021" Jul 13 19:22:35.241105 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:35.241064 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-7m2hd" event={"ID":"1d5a2de8-d8a3-4f9d-a722-7b514344e080","Type":"ContainerStarted","Data":"8a1889e9bed6575f96fc5dcb34d5a5496a5b78b24cf24e9eeced3cc212015bc0"} Jul 13 19:22:35.257869 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:35.257798 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-7m2hd" podStartSLOduration=1.6047838300000001 podStartE2EDuration="4.257752547s" totalImagesPullingTime="2.652968717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:22:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:35.25670355 +0000 UTC m=+50.370611284" watchObservedRunningTime="2026-07-13 19:22:35.257752547 +0000 UTC m=+50.371660256" Jul 13 19:22:36.573919 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:36.573873 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/244c17d0-fb0f-4a75-a802-e99dd9dc3556-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:36.574427 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:36.573975 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/244c17d0-fb0f-4a75-a802-e99dd9dc3556-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:36.574427 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:36.574139 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 19:22:36.574427 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:36.574204 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244c17d0-fb0f-4a75-a802-e99dd9dc3556-insights-runtime-extractor-tls podName:244c17d0-fb0f-4a75-a802-e99dd9dc3556 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:44.574183901 +0000 UTC m=+59.688091595 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/244c17d0-fb0f-4a75-a802-e99dd9dc3556-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r4flg" (UID: "244c17d0-fb0f-4a75-a802-e99dd9dc3556") : secret "insights-runtime-extractor-tls" not found Jul 13 19:22:36.574642 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:36.574554 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/244c17d0-fb0f-4a75-a802-e99dd9dc3556-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:38.689910 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:38.689867 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls\") pod \"dns-default-k24np\" (UID: \"ee1b48b4-0511-4f05-8294-434364af55c1\") " pod="openshift-dns/dns-default-k24np" Jul 13 19:22:38.690390 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:38.689989 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 19:22:38.690390 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:38.690045 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls podName:ee1b48b4-0511-4f05-8294-434364af55c1 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:54.690031358 +0000 UTC m=+69.803939044 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls") pod "dns-default-k24np" (UID: "ee1b48b4-0511-4f05-8294-434364af55c1") : secret "dns-default-metrics-tls" not found Jul 13 19:22:44.628269 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:44.628219 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/244c17d0-fb0f-4a75-a802-e99dd9dc3556-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:44.630804 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:44.630758 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/244c17d0-fb0f-4a75-a802-e99dd9dc3556-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r4flg\" (UID: \"244c17d0-fb0f-4a75-a802-e99dd9dc3556\") " pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:44.722091 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:44.722039 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r4flg" Jul 13 19:22:44.895549 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:44.895457 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r4flg"] Jul 13 19:22:44.899214 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:22:44.899174 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod244c17d0_fb0f_4a75_a802_e99dd9dc3556.slice/crio-389cfc0727bcd605d4f9e8f1380f756e7b24453a393e969a0b953d9894ea0cf1 WatchSource:0}: Error finding container 389cfc0727bcd605d4f9e8f1380f756e7b24453a393e969a0b953d9894ea0cf1: Status 404 returned error can't find the container with id 389cfc0727bcd605d4f9e8f1380f756e7b24453a393e969a0b953d9894ea0cf1 Jul 13 19:22:45.273584 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:45.273497 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r4flg" event={"ID":"244c17d0-fb0f-4a75-a802-e99dd9dc3556","Type":"ContainerStarted","Data":"389cfc0727bcd605d4f9e8f1380f756e7b24453a393e969a0b953d9894ea0cf1"} Jul 13 19:22:48.185491 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:48.185461 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bk46s" Jul 13 19:22:49.287449 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:49.287414 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r4flg" event={"ID":"244c17d0-fb0f-4a75-a802-e99dd9dc3556","Type":"ContainerStarted","Data":"237eb5a3ed1075b2ed3fee4b79dac569cd183cde7a6ff98266fa3c19808b3ea4"} Jul 13 19:22:50.293711 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.293667 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r4flg" event={"ID":"244c17d0-fb0f-4a75-a802-e99dd9dc3556","Type":"ContainerStarted","Data":"22723d2ff7ebe67186250b157e8b1c8293952d53b24b35902bb52e815e7acdc1"} Jul 13 19:22:50.783482 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.783440 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs\") pod \"network-metrics-daemon-sgfvm\" (UID: \"6278283e-2c4b-4f66-915b-934fc03eb4bf\") " pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:22:50.783665 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.783508 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret\") pod \"global-pull-secret-syncer-qgb5b\" (UID: \"5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4\") " pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:22:50.785763 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.785737 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 19:22:50.786144 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.786128 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 19:22:50.796785 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.796750 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4-original-pull-secret\") pod \"global-pull-secret-syncer-qgb5b\" (UID: \"5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4\") " pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:22:50.796955 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.796890 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6278283e-2c4b-4f66-915b-934fc03eb4bf-metrics-certs\") pod \"network-metrics-daemon-sgfvm\" (UID: \"6278283e-2c4b-4f66-915b-934fc03eb4bf\") " pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:22:50.862254 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.862219 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b6d67588f-zwg7r"] Jul 13 19:22:50.865972 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.865600 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:50.869796 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.869757 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 13 19:22:50.869952 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.869758 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 13 19:22:50.870284 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.870135 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 13 19:22:50.870284 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.870136 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-25765\"" Jul 13 19:22:50.872295 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.872260 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fp668\"" Jul 13 19:22:50.877872 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.877846 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 13 19:22:50.879372 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.879348 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgfvm" Jul 13 19:22:50.883516 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.883494 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgb5b" Jul 13 19:22:50.884828 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.884802 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzqnt\" (UniqueName: \"kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt\") pod \"network-check-target-f2lcv\" (UID: \"83623552-aebf-463b-acc9-4cedaa7a7feb\") " pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:50.886947 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.886925 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 19:22:50.895276 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.895143 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b6d67588f-zwg7r"] Jul 13 19:22:50.897075 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.897049 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 19:22:50.908591 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.908560 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzqnt\" (UniqueName: \"kubernetes.io/projected/83623552-aebf-463b-acc9-4cedaa7a7feb-kube-api-access-rzqnt\") pod \"network-check-target-f2lcv\" (UID: \"83623552-aebf-463b-acc9-4cedaa7a7feb\") " pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:50.985434 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.985277 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/af3abc1b-2644-468b-9144-d199a1417e34-ca-trust-extracted\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:50.985434 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.985320 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af3abc1b-2644-468b-9144-d199a1417e34-trusted-ca\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:50.985434 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.985359 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/af3abc1b-2644-468b-9144-d199a1417e34-installation-pull-secrets\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:50.985434 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.985421 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af3abc1b-2644-468b-9144-d199a1417e34-registry-tls\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:50.985802 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.985450 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/af3abc1b-2644-468b-9144-d199a1417e34-registry-certificates\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:50.985802 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.985488 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br4cm\" (UniqueName: \"kubernetes.io/projected/af3abc1b-2644-468b-9144-d199a1417e34-kube-api-access-br4cm\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:50.985802 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.985528 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/af3abc1b-2644-468b-9144-d199a1417e34-image-registry-private-configuration\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:50.985802 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:50.985583 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af3abc1b-2644-468b-9144-d199a1417e34-bound-sa-token\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:51.056105 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.055960 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qgb5b"] Jul 13 19:22:51.060600 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:22:51.060565 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5acbdd58_54bb_4a1e_a631_0ced4ea6c5d4.slice/crio-dd5b227ed960a874cf8e4ab40a059b49406eb54d20e16975b60658c680c2f1fe WatchSource:0}: Error finding container dd5b227ed960a874cf8e4ab40a059b49406eb54d20e16975b60658c680c2f1fe: Status 404 returned error can't find the container with id dd5b227ed960a874cf8e4ab40a059b49406eb54d20e16975b60658c680c2f1fe Jul 13 19:22:51.076219 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.076186 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sgfvm"] Jul 13 19:22:51.079748 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:22:51.079714 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6278283e_2c4b_4f66_915b_934fc03eb4bf.slice/crio-c5a317c3393236431cd7954662dbd3f04272cb2e9bc3c502052aec6878fe8d42 WatchSource:0}: Error finding container c5a317c3393236431cd7954662dbd3f04272cb2e9bc3c502052aec6878fe8d42: Status 404 returned error can't find the container with id c5a317c3393236431cd7954662dbd3f04272cb2e9bc3c502052aec6878fe8d42 Jul 13 19:22:51.087007 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.086978 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/af3abc1b-2644-468b-9144-d199a1417e34-ca-trust-extracted\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:51.087145 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.087021 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af3abc1b-2644-468b-9144-d199a1417e34-trusted-ca\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:51.087145 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.087047 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/af3abc1b-2644-468b-9144-d199a1417e34-installation-pull-secrets\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:51.087145 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.087089 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af3abc1b-2644-468b-9144-d199a1417e34-registry-tls\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:51.087145 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.087121 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/af3abc1b-2644-468b-9144-d199a1417e34-registry-certificates\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:51.087145 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.087147 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br4cm\" (UniqueName: \"kubernetes.io/projected/af3abc1b-2644-468b-9144-d199a1417e34-kube-api-access-br4cm\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:51.087412 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.087179 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/af3abc1b-2644-468b-9144-d199a1417e34-image-registry-private-configuration\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:51.087412 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.087214 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af3abc1b-2644-468b-9144-d199a1417e34-bound-sa-token\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:51.088013 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.087958 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/af3abc1b-2644-468b-9144-d199a1417e34-ca-trust-extracted\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:51.088362 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.088322 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/af3abc1b-2644-468b-9144-d199a1417e34-registry-certificates\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:51.088726 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.088700 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af3abc1b-2644-468b-9144-d199a1417e34-trusted-ca\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:51.090204 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.090175 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/af3abc1b-2644-468b-9144-d199a1417e34-image-registry-private-configuration\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:51.090297 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.090180 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af3abc1b-2644-468b-9144-d199a1417e34-registry-tls\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:51.090297 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.090290 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/af3abc1b-2644-468b-9144-d199a1417e34-installation-pull-secrets\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:51.104984 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.104948 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af3abc1b-2644-468b-9144-d199a1417e34-bound-sa-token\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:51.105175 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.105102 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-br4cm\" (UniqueName: \"kubernetes.io/projected/af3abc1b-2644-468b-9144-d199a1417e34-kube-api-access-br4cm\") pod \"image-registry-5b6d67588f-zwg7r\" (UID: \"af3abc1b-2644-468b-9144-d199a1417e34\") " pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:51.179358 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.179313 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ztp4q\"" Jul 13 19:22:51.180927 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.180896 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:51.187770 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.187735 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:51.300741 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.300700 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sgfvm" event={"ID":"6278283e-2c4b-4f66-915b-934fc03eb4bf","Type":"ContainerStarted","Data":"c5a317c3393236431cd7954662dbd3f04272cb2e9bc3c502052aec6878fe8d42"} Jul 13 19:22:51.302412 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.302324 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qgb5b" event={"ID":"5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4","Type":"ContainerStarted","Data":"dd5b227ed960a874cf8e4ab40a059b49406eb54d20e16975b60658c680c2f1fe"} Jul 13 19:22:51.339483 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.339394 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b6d67588f-zwg7r"] Jul 13 19:22:51.352333 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:51.352292 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f2lcv"] Jul 13 19:22:52.331910 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:22:52.331878 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83623552_aebf_463b_acc9_4cedaa7a7feb.slice/crio-3e20689cdbce76fb774759fa2bcaf75e3e1c0cca48b579aa522db8d28ffe7d7d WatchSource:0}: Error finding container 3e20689cdbce76fb774759fa2bcaf75e3e1c0cca48b579aa522db8d28ffe7d7d: Status 404 returned error can't find the container with id 3e20689cdbce76fb774759fa2bcaf75e3e1c0cca48b579aa522db8d28ffe7d7d Jul 13 19:22:52.332640 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:22:52.332605 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf3abc1b_2644_468b_9144_d199a1417e34.slice/crio-dec8f6071d1c57a88c692d1fdab5a92210d87d3325a8a6c262169265b168bbb5 WatchSource:0}: Error finding container dec8f6071d1c57a88c692d1fdab5a92210d87d3325a8a6c262169265b168bbb5: Status 404 returned error can't find the container with id dec8f6071d1c57a88c692d1fdab5a92210d87d3325a8a6c262169265b168bbb5 Jul 13 19:22:53.323656 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:53.323408 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" event={"ID":"af3abc1b-2644-468b-9144-d199a1417e34","Type":"ContainerStarted","Data":"a9931a464e240bf904cba243f60615f2cb1b8c4b6016a041efe4ca51507dced7"} Jul 13 19:22:53.323656 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:53.323516 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" event={"ID":"af3abc1b-2644-468b-9144-d199a1417e34","Type":"ContainerStarted","Data":"dec8f6071d1c57a88c692d1fdab5a92210d87d3325a8a6c262169265b168bbb5"} Jul 13 19:22:53.323656 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:53.323544 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:22:53.325511 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:53.325469 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f2lcv" event={"ID":"83623552-aebf-463b-acc9-4cedaa7a7feb","Type":"ContainerStarted","Data":"3e20689cdbce76fb774759fa2bcaf75e3e1c0cca48b579aa522db8d28ffe7d7d"} Jul 13 19:22:53.327868 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:53.327828 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sgfvm" event={"ID":"6278283e-2c4b-4f66-915b-934fc03eb4bf","Type":"ContainerStarted","Data":"90dc7c518d804b78d39ea372780f74a0c10cd6b2ea874d243325a0a8411a562e"} Jul 13 19:22:53.328039 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:53.327903 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sgfvm" event={"ID":"6278283e-2c4b-4f66-915b-934fc03eb4bf","Type":"ContainerStarted","Data":"f618858893db416d33ca50d3e9afaf31f09b36a204ca9305e7ac6f9792288a95"} Jul 13 19:22:53.330704 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:53.330674 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r4flg" event={"ID":"244c17d0-fb0f-4a75-a802-e99dd9dc3556","Type":"ContainerStarted","Data":"42af717cedbf6adc00f39f6e5a5b138548d3485a8a9e1f8b56afcdd319d35955"} Jul 13 19:22:53.343048 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:53.342989 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" podStartSLOduration=3.342961693 podStartE2EDuration="3.342961693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:22:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:53.341808281 +0000 UTC m=+68.455715991" watchObservedRunningTime="2026-07-13 19:22:53.342961693 +0000 UTC m=+68.456869403" Jul 13 19:22:53.359644 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:53.359567 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sgfvm" podStartSLOduration=65.463763934 podStartE2EDuration="1m7.359539251s" totalImagesPullingTime="1.895775317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:53.357515171 +0000 UTC m=+68.471422881" watchObservedRunningTime="2026-07-13 19:22:53.359539251 +0000 UTC m=+68.473446960" Jul 13 19:22:53.380379 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:53.380291 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-r4flg" podStartSLOduration=18.029610499 podStartE2EDuration="25.380266643s" totalImagesPullingTime="7.350656144s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:22:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:53.379588412 +0000 UTC m=+68.493496123" watchObservedRunningTime="2026-07-13 19:22:53.380266643 +0000 UTC m=+68.494174352" Jul 13 19:22:54.721864 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:54.721820 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls\") pod \"dns-default-k24np\" (UID: \"ee1b48b4-0511-4f05-8294-434364af55c1\") " pod="openshift-dns/dns-default-k24np" Jul 13 19:22:54.724848 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:54.724815 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee1b48b4-0511-4f05-8294-434364af55c1-metrics-tls\") pod \"dns-default-k24np\" (UID: \"ee1b48b4-0511-4f05-8294-434364af55c1\") " pod="openshift-dns/dns-default-k24np" Jul 13 19:22:54.927852 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:54.927813 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mf9hk\"" Jul 13 19:22:54.935635 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:54.935598 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k24np" Jul 13 19:22:57.487540 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:57.487493 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-k24np"] Jul 13 19:22:57.491280 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:22:57.491240 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee1b48b4_0511_4f05_8294_434364af55c1.slice/crio-18985bdddb569885c3871c0bd7bda04adf4be34e82bee3d7a211585362386a80 WatchSource:0}: Error finding container 18985bdddb569885c3871c0bd7bda04adf4be34e82bee3d7a211585362386a80: Status 404 returned error can't find the container with id 18985bdddb569885c3871c0bd7bda04adf4be34e82bee3d7a211585362386a80 Jul 13 19:22:58.354252 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:58.354210 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qgb5b" event={"ID":"5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4","Type":"ContainerStarted","Data":"5adafe3ed58434f009d2bba76d7a9cf5f8f51baf00c338f2fdd2b03c420a706a"} Jul 13 19:22:58.355424 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:58.355397 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k24np" event={"ID":"ee1b48b4-0511-4f05-8294-434364af55c1","Type":"ContainerStarted","Data":"18985bdddb569885c3871c0bd7bda04adf4be34e82bee3d7a211585362386a80"} Jul 13 19:22:58.356911 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:58.356887 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f2lcv" event={"ID":"83623552-aebf-463b-acc9-4cedaa7a7feb","Type":"ContainerStarted","Data":"ed5636d4e2aede30d5056b3149e1be883961a6b2f9e43ea7c259ecd539f960c2"} Jul 13 19:22:58.357044 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:58.357025 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:22:58.374613 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:58.374555 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qgb5b" podStartSLOduration=66.111983233 podStartE2EDuration="1m12.374536753s" totalImagesPullingTime="6.26255352s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:21:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:58.373665465 +0000 UTC m=+73.487573194" watchObservedRunningTime="2026-07-13 19:22:58.374536753 +0000 UTC m=+73.488444461" Jul 13 19:22:58.393508 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:58.393450 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-f2lcv" podStartSLOduration=67.387509653 podStartE2EDuration="1m12.393431663s" totalImagesPullingTime="5.00592201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:21:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:22:58.39268076 +0000 UTC m=+73.506588470" watchObservedRunningTime="2026-07-13 19:22:58.393431663 +0000 UTC m=+73.507339382" Jul 13 19:22:58.573715 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:58.573676 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xlhtc"] Jul 13 19:22:58.595402 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:58.595342 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xlhtc"] Jul 13 19:22:58.595606 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:58.595475 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xlhtc" Jul 13 19:22:58.597535 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:58.597507 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 13 19:22:58.597702 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:58.597675 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-nbccr\"" Jul 13 19:22:58.649345 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:58.649255 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7b87aa78-a616-4642-a245-108f3baabde8-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-xlhtc\" (UID: \"7b87aa78-a616-4642-a245-108f3baabde8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xlhtc" Jul 13 19:22:58.749823 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:58.749767 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7b87aa78-a616-4642-a245-108f3baabde8-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-xlhtc\" (UID: \"7b87aa78-a616-4642-a245-108f3baabde8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xlhtc" Jul 13 19:22:58.750053 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:58.749901 2556 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 13 19:22:58.750053 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:22:58.749981 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b87aa78-a616-4642-a245-108f3baabde8-tls-certificates podName:7b87aa78-a616-4642-a245-108f3baabde8 nodeName:}" failed. No retries permitted until 2026-07-13 19:22:59.249953026 +0000 UTC m=+74.363860736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/7b87aa78-a616-4642-a245-108f3baabde8-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-xlhtc" (UID: "7b87aa78-a616-4642-a245-108f3baabde8") : secret "prometheus-operator-admission-webhook-tls" not found Jul 13 19:22:59.254973 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:59.254934 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7b87aa78-a616-4642-a245-108f3baabde8-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-xlhtc\" (UID: \"7b87aa78-a616-4642-a245-108f3baabde8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xlhtc" Jul 13 19:22:59.257816 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:59.257767 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7b87aa78-a616-4642-a245-108f3baabde8-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-xlhtc\" (UID: \"7b87aa78-a616-4642-a245-108f3baabde8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xlhtc" Jul 13 19:22:59.507651 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:59.507482 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xlhtc" Jul 13 19:22:59.686536 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:22:59.686506 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xlhtc"] Jul 13 19:22:59.690589 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:22:59.690560 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b87aa78_a616_4642_a245_108f3baabde8.slice/crio-f5b54acb0e80866913d009a85a1f4ae1e85801657195ba2dcd0aecdff5f36576 WatchSource:0}: Error finding container f5b54acb0e80866913d009a85a1f4ae1e85801657195ba2dcd0aecdff5f36576: Status 404 returned error can't find the container with id f5b54acb0e80866913d009a85a1f4ae1e85801657195ba2dcd0aecdff5f36576 Jul 13 19:23:00.367554 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:00.367499 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k24np" event={"ID":"ee1b48b4-0511-4f05-8294-434364af55c1","Type":"ContainerStarted","Data":"7ed643060faef5160c90e5f922399c3a58241f46ab939151de958a840d9e99ae"} Jul 13 19:23:00.367795 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:00.367590 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-k24np" Jul 13 19:23:00.367795 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:00.367611 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k24np" event={"ID":"ee1b48b4-0511-4f05-8294-434364af55c1","Type":"ContainerStarted","Data":"a5760ef5d139e3e79f04500cc5eeee5e8b9312f9ca9606776672577228fd72b3"} Jul 13 19:23:00.369163 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:00.369133 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xlhtc" event={"ID":"7b87aa78-a616-4642-a245-108f3baabde8","Type":"ContainerStarted","Data":"f5b54acb0e80866913d009a85a1f4ae1e85801657195ba2dcd0aecdff5f36576"} Jul 13 19:23:00.383994 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:00.383927 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-k24np" podStartSLOduration=36.299346797 podStartE2EDuration="38.383901708s" totalImagesPullingTime="2.084554911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:22:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:23:00.382833056 +0000 UTC m=+75.496740767" watchObservedRunningTime="2026-07-13 19:23:00.383901708 +0000 UTC m=+75.497809435" Jul 13 19:23:01.375858 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.375819 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xlhtc" event={"ID":"7b87aa78-a616-4642-a245-108f3baabde8","Type":"ContainerStarted","Data":"4d6c41d7b7a7b34e4b75bf7a33321ad3118eeed0df657ff9c3df874b36a93a79"} Jul 13 19:23:01.376293 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.375981 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xlhtc" Jul 13 19:23:01.380667 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.380640 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xlhtc" Jul 13 19:23:01.391373 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.391315 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xlhtc" podStartSLOduration=2.20990602 podStartE2EDuration="3.391298192s" totalImagesPullingTime="1.181392172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:22:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:23:01.390161794 +0000 UTC m=+76.504069503" watchObservedRunningTime="2026-07-13 19:23:01.391298192 +0000 UTC m=+76.505205901" Jul 13 19:23:01.689662 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.689580 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-knl5f"] Jul 13 19:23:01.691760 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.691738 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-knl5f" Jul 13 19:23:01.696320 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.696293 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 19:23:01.696320 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.696293 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 13 19:23:01.696685 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.696669 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 19:23:01.696685 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.696676 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 19:23:01.697014 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.696997 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-7ldff\"" Jul 13 19:23:01.698125 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.698105 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 13 19:23:01.709675 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.709643 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-knl5f"] Jul 13 19:23:01.773102 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.773062 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e2d8ed68-6996-4c40-9f6a-7a03384fb5b1-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-knl5f\" (UID: \"e2d8ed68-6996-4c40-9f6a-7a03384fb5b1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-knl5f" Jul 13 19:23:01.773102 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.773106 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2d8ed68-6996-4c40-9f6a-7a03384fb5b1-metrics-client-ca\") pod \"prometheus-operator-64785b6675-knl5f\" (UID: \"e2d8ed68-6996-4c40-9f6a-7a03384fb5b1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-knl5f" Jul 13 19:23:01.773335 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.773168 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7m8c\" (UniqueName: \"kubernetes.io/projected/e2d8ed68-6996-4c40-9f6a-7a03384fb5b1-kube-api-access-p7m8c\") pod \"prometheus-operator-64785b6675-knl5f\" (UID: \"e2d8ed68-6996-4c40-9f6a-7a03384fb5b1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-knl5f" Jul 13 19:23:01.773335 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.773217 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2d8ed68-6996-4c40-9f6a-7a03384fb5b1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-knl5f\" (UID: \"e2d8ed68-6996-4c40-9f6a-7a03384fb5b1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-knl5f" Jul 13 19:23:01.874258 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.874221 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7m8c\" (UniqueName: \"kubernetes.io/projected/e2d8ed68-6996-4c40-9f6a-7a03384fb5b1-kube-api-access-p7m8c\") pod \"prometheus-operator-64785b6675-knl5f\" (UID: \"e2d8ed68-6996-4c40-9f6a-7a03384fb5b1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-knl5f" Jul 13 19:23:01.874386 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.874267 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2d8ed68-6996-4c40-9f6a-7a03384fb5b1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-knl5f\" (UID: \"e2d8ed68-6996-4c40-9f6a-7a03384fb5b1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-knl5f" Jul 13 19:23:01.874386 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.874321 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e2d8ed68-6996-4c40-9f6a-7a03384fb5b1-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-knl5f\" (UID: \"e2d8ed68-6996-4c40-9f6a-7a03384fb5b1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-knl5f" Jul 13 19:23:01.874386 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.874343 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2d8ed68-6996-4c40-9f6a-7a03384fb5b1-metrics-client-ca\") pod \"prometheus-operator-64785b6675-knl5f\" (UID: \"e2d8ed68-6996-4c40-9f6a-7a03384fb5b1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-knl5f" Jul 13 19:23:01.875204 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.875176 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2d8ed68-6996-4c40-9f6a-7a03384fb5b1-metrics-client-ca\") pod \"prometheus-operator-64785b6675-knl5f\" (UID: \"e2d8ed68-6996-4c40-9f6a-7a03384fb5b1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-knl5f" Jul 13 19:23:01.876956 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.876927 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2d8ed68-6996-4c40-9f6a-7a03384fb5b1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-knl5f\" (UID: \"e2d8ed68-6996-4c40-9f6a-7a03384fb5b1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-knl5f" Jul 13 19:23:01.877146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.877123 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e2d8ed68-6996-4c40-9f6a-7a03384fb5b1-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-knl5f\" (UID: \"e2d8ed68-6996-4c40-9f6a-7a03384fb5b1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-knl5f" Jul 13 19:23:01.884205 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:01.884175 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7m8c\" (UniqueName: \"kubernetes.io/projected/e2d8ed68-6996-4c40-9f6a-7a03384fb5b1-kube-api-access-p7m8c\") pod \"prometheus-operator-64785b6675-knl5f\" (UID: \"e2d8ed68-6996-4c40-9f6a-7a03384fb5b1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-knl5f" Jul 13 19:23:02.002467 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:02.002369 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-knl5f" Jul 13 19:23:02.131971 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:02.131937 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-knl5f"] Jul 13 19:23:02.135407 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:23:02.135371 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2d8ed68_6996_4c40_9f6a_7a03384fb5b1.slice/crio-6ffb1996fa0cfcb6f77f658a8fc3dc2de94b64f663915710840abac8f03030d3 WatchSource:0}: Error finding container 6ffb1996fa0cfcb6f77f658a8fc3dc2de94b64f663915710840abac8f03030d3: Status 404 returned error can't find the container with id 6ffb1996fa0cfcb6f77f658a8fc3dc2de94b64f663915710840abac8f03030d3 Jul 13 19:23:02.380950 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:02.380911 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-knl5f" event={"ID":"e2d8ed68-6996-4c40-9f6a-7a03384fb5b1","Type":"ContainerStarted","Data":"6ffb1996fa0cfcb6f77f658a8fc3dc2de94b64f663915710840abac8f03030d3"} Jul 13 19:23:04.391458 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:04.391416 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-knl5f" event={"ID":"e2d8ed68-6996-4c40-9f6a-7a03384fb5b1","Type":"ContainerStarted","Data":"2022ab50dcdf21201277327e4548cfcb0df148f73e3234d65af16b4f18da3db9"} Jul 13 19:23:04.391958 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:04.391513 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-knl5f" event={"ID":"e2d8ed68-6996-4c40-9f6a-7a03384fb5b1","Type":"ContainerStarted","Data":"d23f2ff8918493d0c9ffe26b0a980ed3d7b63397f2d73a04b1cde04872ca999d"} Jul 13 19:23:04.407280 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:04.407216 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-knl5f" podStartSLOduration=2.114890773 podStartE2EDuration="3.407198139s" totalImagesPullingTime="1.292307366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:23:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:23:04.406758616 +0000 UTC m=+79.520666327" watchObservedRunningTime="2026-07-13 19:23:04.407198139 +0000 UTC m=+79.521105847" Jul 13 19:23:06.050564 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.050528 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn"] Jul 13 19:23:06.052850 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.052830 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" Jul 13 19:23:06.054743 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.054717 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 13 19:23:06.055167 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.055143 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 13 19:23:06.055273 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.055143 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-rq9nt\"" Jul 13 19:23:06.064142 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.064103 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn"] Jul 13 19:23:06.076399 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.076239 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-c8pz6"] Jul 13 19:23:06.078785 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.078754 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.080753 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.080725 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 13 19:23:06.080939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.080727 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-l94gb\"" Jul 13 19:23:06.081003 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.080728 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 13 19:23:06.081272 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.081255 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 13 19:23:06.092030 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.092000 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qg567"] Jul 13 19:23:06.098897 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.098844 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-c8pz6"] Jul 13 19:23:06.099075 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.099029 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.101462 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.101425 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39da47d2-4e42-4d22-955e-10adfc404b57-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-2gxgn\" (UID: \"39da47d2-4e42-4d22-955e-10adfc404b57\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" Jul 13 19:23:06.101599 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.101487 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39da47d2-4e42-4d22-955e-10adfc404b57-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-2gxgn\" (UID: \"39da47d2-4e42-4d22-955e-10adfc404b57\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" Jul 13 19:23:06.101599 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.101529 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39da47d2-4e42-4d22-955e-10adfc404b57-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-2gxgn\" (UID: \"39da47d2-4e42-4d22-955e-10adfc404b57\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" Jul 13 19:23:06.101599 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.101576 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgh8m\" (UniqueName: \"kubernetes.io/projected/39da47d2-4e42-4d22-955e-10adfc404b57-kube-api-access-cgh8m\") pod \"openshift-state-metrics-84bd68b896-2gxgn\" (UID: \"39da47d2-4e42-4d22-955e-10adfc404b57\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" Jul 13 19:23:06.102987 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.102737 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 19:23:06.102987 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.102817 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 19:23:06.103191 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.103052 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 19:23:06.103413 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.103390 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gdkc8\"" Jul 13 19:23:06.202493 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.202453 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c04a5ce2-c2ae-4036-9a7c-519c30553a48-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-c8pz6\" (UID: \"c04a5ce2-c2ae-4036-9a7c-519c30553a48\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.202688 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.202503 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-624rf\" (UniqueName: \"kubernetes.io/projected/c04a5ce2-c2ae-4036-9a7c-519c30553a48-kube-api-access-624rf\") pod \"kube-state-metrics-785778ff74-c8pz6\" (UID: \"c04a5ce2-c2ae-4036-9a7c-519c30553a48\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.202688 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.202543 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39da47d2-4e42-4d22-955e-10adfc404b57-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-2gxgn\" (UID: \"39da47d2-4e42-4d22-955e-10adfc404b57\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" Jul 13 19:23:06.202688 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.202576 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39da47d2-4e42-4d22-955e-10adfc404b57-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-2gxgn\" (UID: \"39da47d2-4e42-4d22-955e-10adfc404b57\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" Jul 13 19:23:06.202688 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.202605 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4aa6015-fd26-40b5-abfe-d3df70e1c058-sys\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.202688 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.202629 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e4aa6015-fd26-40b5-abfe-d3df70e1c058-node-exporter-wtmp\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.202688 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.202660 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39da47d2-4e42-4d22-955e-10adfc404b57-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-2gxgn\" (UID: \"39da47d2-4e42-4d22-955e-10adfc404b57\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" Jul 13 19:23:06.202988 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.202727 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e4aa6015-fd26-40b5-abfe-d3df70e1c058-node-exporter-tls\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.202988 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.202828 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c04a5ce2-c2ae-4036-9a7c-519c30553a48-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-c8pz6\" (UID: \"c04a5ce2-c2ae-4036-9a7c-519c30553a48\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.202988 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:23:06.202860 2556 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 13 19:23:06.202988 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:23:06.202951 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39da47d2-4e42-4d22-955e-10adfc404b57-openshift-state-metrics-tls podName:39da47d2-4e42-4d22-955e-10adfc404b57 nodeName:}" failed. No retries permitted until 2026-07-13 19:23:06.702926039 +0000 UTC m=+81.816833746 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/39da47d2-4e42-4d22-955e-10adfc404b57-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-2gxgn" (UID: "39da47d2-4e42-4d22-955e-10adfc404b57") : secret "openshift-state-metrics-tls" not found Jul 13 19:23:06.202988 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.202863 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e4aa6015-fd26-40b5-abfe-d3df70e1c058-root\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.203237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.203001 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e4aa6015-fd26-40b5-abfe-d3df70e1c058-node-exporter-textfile\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.203237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.203048 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgh8m\" (UniqueName: \"kubernetes.io/projected/39da47d2-4e42-4d22-955e-10adfc404b57-kube-api-access-cgh8m\") pod \"openshift-state-metrics-84bd68b896-2gxgn\" (UID: \"39da47d2-4e42-4d22-955e-10adfc404b57\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" Jul 13 19:23:06.203237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.203075 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4aa6015-fd26-40b5-abfe-d3df70e1c058-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.203237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.203107 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4aa6015-fd26-40b5-abfe-d3df70e1c058-metrics-client-ca\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.203237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.203138 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c04a5ce2-c2ae-4036-9a7c-519c30553a48-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-c8pz6\" (UID: \"c04a5ce2-c2ae-4036-9a7c-519c30553a48\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.203237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.203182 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e4aa6015-fd26-40b5-abfe-d3df70e1c058-node-exporter-accelerators-collector-config\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.203237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.203213 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c04a5ce2-c2ae-4036-9a7c-519c30553a48-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-c8pz6\" (UID: \"c04a5ce2-c2ae-4036-9a7c-519c30553a48\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.203553 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.203258 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c04a5ce2-c2ae-4036-9a7c-519c30553a48-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-c8pz6\" (UID: \"c04a5ce2-c2ae-4036-9a7c-519c30553a48\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.203553 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.203284 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd8nc\" (UniqueName: \"kubernetes.io/projected/e4aa6015-fd26-40b5-abfe-d3df70e1c058-kube-api-access-dd8nc\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.203553 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.203418 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39da47d2-4e42-4d22-955e-10adfc404b57-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-2gxgn\" (UID: \"39da47d2-4e42-4d22-955e-10adfc404b57\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" Jul 13 19:23:06.205366 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.205339 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39da47d2-4e42-4d22-955e-10adfc404b57-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-2gxgn\" (UID: \"39da47d2-4e42-4d22-955e-10adfc404b57\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" Jul 13 19:23:06.215027 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.214999 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgh8m\" (UniqueName: \"kubernetes.io/projected/39da47d2-4e42-4d22-955e-10adfc404b57-kube-api-access-cgh8m\") pod \"openshift-state-metrics-84bd68b896-2gxgn\" (UID: \"39da47d2-4e42-4d22-955e-10adfc404b57\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" Jul 13 19:23:06.304237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.304138 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e4aa6015-fd26-40b5-abfe-d3df70e1c058-node-exporter-tls\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.304237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.304186 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c04a5ce2-c2ae-4036-9a7c-519c30553a48-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-c8pz6\" (UID: \"c04a5ce2-c2ae-4036-9a7c-519c30553a48\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.304237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.304211 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e4aa6015-fd26-40b5-abfe-d3df70e1c058-root\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.304237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.304229 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e4aa6015-fd26-40b5-abfe-d3df70e1c058-node-exporter-textfile\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.304542 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.304301 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e4aa6015-fd26-40b5-abfe-d3df70e1c058-root\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.304542 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.304349 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4aa6015-fd26-40b5-abfe-d3df70e1c058-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.304542 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.304382 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4aa6015-fd26-40b5-abfe-d3df70e1c058-metrics-client-ca\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.304542 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.304405 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c04a5ce2-c2ae-4036-9a7c-519c30553a48-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-c8pz6\" (UID: \"c04a5ce2-c2ae-4036-9a7c-519c30553a48\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.304542 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.304435 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e4aa6015-fd26-40b5-abfe-d3df70e1c058-node-exporter-accelerators-collector-config\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.304800 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.304612 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c04a5ce2-c2ae-4036-9a7c-519c30553a48-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-c8pz6\" (UID: \"c04a5ce2-c2ae-4036-9a7c-519c30553a48\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.304800 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.304623 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e4aa6015-fd26-40b5-abfe-d3df70e1c058-node-exporter-textfile\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.304800 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.304686 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c04a5ce2-c2ae-4036-9a7c-519c30553a48-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-c8pz6\" (UID: \"c04a5ce2-c2ae-4036-9a7c-519c30553a48\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.304800 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.304717 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd8nc\" (UniqueName: \"kubernetes.io/projected/e4aa6015-fd26-40b5-abfe-d3df70e1c058-kube-api-access-dd8nc\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.304800 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.304766 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c04a5ce2-c2ae-4036-9a7c-519c30553a48-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-c8pz6\" (UID: \"c04a5ce2-c2ae-4036-9a7c-519c30553a48\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.305060 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.304838 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-624rf\" (UniqueName: \"kubernetes.io/projected/c04a5ce2-c2ae-4036-9a7c-519c30553a48-kube-api-access-624rf\") pod \"kube-state-metrics-785778ff74-c8pz6\" (UID: \"c04a5ce2-c2ae-4036-9a7c-519c30553a48\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.305060 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.304879 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4aa6015-fd26-40b5-abfe-d3df70e1c058-sys\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.305060 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.304904 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e4aa6015-fd26-40b5-abfe-d3df70e1c058-node-exporter-wtmp\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.305208 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.305057 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e4aa6015-fd26-40b5-abfe-d3df70e1c058-node-exporter-accelerators-collector-config\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.305208 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.305057 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4aa6015-fd26-40b5-abfe-d3df70e1c058-metrics-client-ca\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.305208 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.305107 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e4aa6015-fd26-40b5-abfe-d3df70e1c058-node-exporter-wtmp\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.305208 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.305126 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c04a5ce2-c2ae-4036-9a7c-519c30553a48-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-c8pz6\" (UID: \"c04a5ce2-c2ae-4036-9a7c-519c30553a48\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.305404 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.305331 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4aa6015-fd26-40b5-abfe-d3df70e1c058-sys\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.305623 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.305564 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c04a5ce2-c2ae-4036-9a7c-519c30553a48-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-c8pz6\" (UID: \"c04a5ce2-c2ae-4036-9a7c-519c30553a48\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.305732 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.305673 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c04a5ce2-c2ae-4036-9a7c-519c30553a48-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-c8pz6\" (UID: \"c04a5ce2-c2ae-4036-9a7c-519c30553a48\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.307041 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.307013 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4aa6015-fd26-40b5-abfe-d3df70e1c058-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.307337 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.307314 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e4aa6015-fd26-40b5-abfe-d3df70e1c058-node-exporter-tls\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.307441 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.307426 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c04a5ce2-c2ae-4036-9a7c-519c30553a48-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-c8pz6\" (UID: \"c04a5ce2-c2ae-4036-9a7c-519c30553a48\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.307489 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.307462 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c04a5ce2-c2ae-4036-9a7c-519c30553a48-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-c8pz6\" (UID: \"c04a5ce2-c2ae-4036-9a7c-519c30553a48\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.313351 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.313325 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd8nc\" (UniqueName: \"kubernetes.io/projected/e4aa6015-fd26-40b5-abfe-d3df70e1c058-kube-api-access-dd8nc\") pod \"node-exporter-qg567\" (UID: \"e4aa6015-fd26-40b5-abfe-d3df70e1c058\") " pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.313461 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.313395 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-624rf\" (UniqueName: \"kubernetes.io/projected/c04a5ce2-c2ae-4036-9a7c-519c30553a48-kube-api-access-624rf\") pod \"kube-state-metrics-785778ff74-c8pz6\" (UID: \"c04a5ce2-c2ae-4036-9a7c-519c30553a48\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.390554 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.390431 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" Jul 13 19:23:06.413345 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.413312 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qg567" Jul 13 19:23:06.425389 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:23:06.425354 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4aa6015_fd26_40b5_abfe_d3df70e1c058.slice/crio-83e16679a6a8ed957f4047b2ac5daafa1e39a04fb7269b1ebd8e1843026e5583 WatchSource:0}: Error finding container 83e16679a6a8ed957f4047b2ac5daafa1e39a04fb7269b1ebd8e1843026e5583: Status 404 returned error can't find the container with id 83e16679a6a8ed957f4047b2ac5daafa1e39a04fb7269b1ebd8e1843026e5583 Jul 13 19:23:06.549541 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.549503 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-c8pz6"] Jul 13 19:23:06.552431 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:23:06.552402 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc04a5ce2_c2ae_4036_9a7c_519c30553a48.slice/crio-dde5a7969b456daed83a537c5ac936977eb8464796443c29daeb6e2e0b972540 WatchSource:0}: Error finding container dde5a7969b456daed83a537c5ac936977eb8464796443c29daeb6e2e0b972540: Status 404 returned error can't find the container with id dde5a7969b456daed83a537c5ac936977eb8464796443c29daeb6e2e0b972540 Jul 13 19:23:06.708372 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.708332 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39da47d2-4e42-4d22-955e-10adfc404b57-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-2gxgn\" (UID: \"39da47d2-4e42-4d22-955e-10adfc404b57\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" Jul 13 19:23:06.710832 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.710799 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39da47d2-4e42-4d22-955e-10adfc404b57-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-2gxgn\" (UID: \"39da47d2-4e42-4d22-955e-10adfc404b57\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" Jul 13 19:23:06.964254 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:06.964147 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" Jul 13 19:23:07.142417 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.142384 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 19:23:07.145497 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:23:07.145470 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39da47d2_4e42_4d22_955e_10adfc404b57.slice/crio-167e83930b58a3d09fa65d89d8a61751252a99a553cc7275d4df08a37b3c210c WatchSource:0}: Error finding container 167e83930b58a3d09fa65d89d8a61751252a99a553cc7275d4df08a37b3c210c: Status 404 returned error can't find the container with id 167e83930b58a3d09fa65d89d8a61751252a99a553cc7275d4df08a37b3c210c Jul 13 19:23:07.146922 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.146279 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn"] Jul 13 19:23:07.146922 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.146439 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.149086 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.149063 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 13 19:23:07.149912 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.149889 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 13 19:23:07.150139 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.150121 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 13 19:23:07.150248 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.150208 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 13 19:23:07.150317 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.150255 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fms67\"" Jul 13 19:23:07.150390 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.150375 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 13 19:23:07.150549 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.150508 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 13 19:23:07.150549 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.150550 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 13 19:23:07.150696 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.150629 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 13 19:23:07.150853 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.150810 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 13 19:23:07.163821 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.163766 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 19:23:07.211973 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.211941 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57caca18-ea13-445b-a21a-016bb5a0f9c0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.212135 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.211999 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-web-config\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.212243 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.212223 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.212307 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.212283 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-config-volume\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.212345 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.212318 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.212388 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.212348 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.212505 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.212484 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6qxt\" (UniqueName: \"kubernetes.io/projected/57caca18-ea13-445b-a21a-016bb5a0f9c0-kube-api-access-v6qxt\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.212571 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.212537 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57caca18-ea13-445b-a21a-016bb5a0f9c0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.212628 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.212584 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.212628 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.212617 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57caca18-ea13-445b-a21a-016bb5a0f9c0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.212726 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.212666 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57caca18-ea13-445b-a21a-016bb5a0f9c0-config-out\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.212726 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.212703 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57caca18-ea13-445b-a21a-016bb5a0f9c0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.212838 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.212743 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.313237 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.313191 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-config-volume\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.313436 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.313253 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.313436 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.313295 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.313436 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.313353 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6qxt\" (UniqueName: \"kubernetes.io/projected/57caca18-ea13-445b-a21a-016bb5a0f9c0-kube-api-access-v6qxt\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.313436 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.313382 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57caca18-ea13-445b-a21a-016bb5a0f9c0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.313436 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.313433 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.313663 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.313464 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57caca18-ea13-445b-a21a-016bb5a0f9c0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.313663 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.313494 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57caca18-ea13-445b-a21a-016bb5a0f9c0-config-out\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.313663 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.313522 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57caca18-ea13-445b-a21a-016bb5a0f9c0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.313663 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.313564 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.313663 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.313598 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57caca18-ea13-445b-a21a-016bb5a0f9c0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.313663 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.313629 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-web-config\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.313663 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.313660 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.315241 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.315047 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57caca18-ea13-445b-a21a-016bb5a0f9c0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.315241 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.315193 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57caca18-ea13-445b-a21a-016bb5a0f9c0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.316658 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.316630 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57caca18-ea13-445b-a21a-016bb5a0f9c0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.317199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.316976 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.317199 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.317068 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.318442 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.318151 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-config-volume\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.318442 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.318403 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.318893 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.318853 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.319294 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.319250 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.319400 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.319358 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57caca18-ea13-445b-a21a-016bb5a0f9c0-config-out\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.319708 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.319659 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57caca18-ea13-445b-a21a-016bb5a0f9c0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.320980 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.320938 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-web-config\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.322150 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.322128 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6qxt\" (UniqueName: \"kubernetes.io/projected/57caca18-ea13-445b-a21a-016bb5a0f9c0-kube-api-access-v6qxt\") pod \"alertmanager-main-0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.404320 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.404273 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" event={"ID":"39da47d2-4e42-4d22-955e-10adfc404b57","Type":"ContainerStarted","Data":"6b81169b9d4dafaf4c1d7957c0756af77be4775f7367207f9b435977cc9699d9"} Jul 13 19:23:07.404617 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.404351 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" event={"ID":"39da47d2-4e42-4d22-955e-10adfc404b57","Type":"ContainerStarted","Data":"167e83930b58a3d09fa65d89d8a61751252a99a553cc7275d4df08a37b3c210c"} Jul 13 19:23:07.405950 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.405906 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" event={"ID":"c04a5ce2-c2ae-4036-9a7c-519c30553a48","Type":"ContainerStarted","Data":"dde5a7969b456daed83a537c5ac936977eb8464796443c29daeb6e2e0b972540"} Jul 13 19:23:07.407518 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.407486 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qg567" event={"ID":"e4aa6015-fd26-40b5-abfe-d3df70e1c058","Type":"ContainerStarted","Data":"83e16679a6a8ed957f4047b2ac5daafa1e39a04fb7269b1ebd8e1843026e5583"} Jul 13 19:23:07.459914 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.459875 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:23:07.620547 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:07.620511 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 19:23:07.658989 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:23:07.658946 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57caca18_ea13_445b_a21a_016bb5a0f9c0.slice/crio-7562ee0c27032504efd0b520be89a667140a35725d1fbc5523df9a6ead863bd5 WatchSource:0}: Error finding container 7562ee0c27032504efd0b520be89a667140a35725d1fbc5523df9a6ead863bd5: Status 404 returned error can't find the container with id 7562ee0c27032504efd0b520be89a667140a35725d1fbc5523df9a6ead863bd5 Jul 13 19:23:08.122376 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.122289 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7d9499477c-2lz6w"] Jul 13 19:23:08.125224 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.125197 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.127574 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.127546 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 13 19:23:08.127739 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.127545 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 13 19:23:08.127739 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.127546 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 13 19:23:08.127739 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.127611 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-5v86w\"" Jul 13 19:23:08.127739 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.127704 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 13 19:23:08.127739 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.127546 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6858744rkddng\"" Jul 13 19:23:08.128035 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.127960 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 13 19:23:08.137239 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.137212 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d9499477c-2lz6w"] Jul 13 19:23:08.221972 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.221920 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/08923444-e4b3-4cdb-8bba-fe70a9b5c218-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.221972 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.221972 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/08923444-e4b3-4cdb-8bba-fe70a9b5c218-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.222391 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.222005 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08923444-e4b3-4cdb-8bba-fe70a9b5c218-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.222391 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.222032 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc82m\" (UniqueName: \"kubernetes.io/projected/08923444-e4b3-4cdb-8bba-fe70a9b5c218-kube-api-access-cc82m\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.222391 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.222058 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/08923444-e4b3-4cdb-8bba-fe70a9b5c218-secret-thanos-querier-tls\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.222391 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.222106 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/08923444-e4b3-4cdb-8bba-fe70a9b5c218-secret-grpc-tls\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.222391 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.222139 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08923444-e4b3-4cdb-8bba-fe70a9b5c218-metrics-client-ca\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.222391 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.222192 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08923444-e4b3-4cdb-8bba-fe70a9b5c218-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.323450 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.323405 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/08923444-e4b3-4cdb-8bba-fe70a9b5c218-secret-grpc-tls\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.323667 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.323470 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08923444-e4b3-4cdb-8bba-fe70a9b5c218-metrics-client-ca\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.323729 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.323687 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08923444-e4b3-4cdb-8bba-fe70a9b5c218-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.323814 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.323792 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/08923444-e4b3-4cdb-8bba-fe70a9b5c218-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.323879 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.323841 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/08923444-e4b3-4cdb-8bba-fe70a9b5c218-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.323941 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.323876 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08923444-e4b3-4cdb-8bba-fe70a9b5c218-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.323941 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.323909 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc82m\" (UniqueName: \"kubernetes.io/projected/08923444-e4b3-4cdb-8bba-fe70a9b5c218-kube-api-access-cc82m\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.324030 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.323953 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/08923444-e4b3-4cdb-8bba-fe70a9b5c218-secret-thanos-querier-tls\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.324723 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.324363 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08923444-e4b3-4cdb-8bba-fe70a9b5c218-metrics-client-ca\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.327060 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.327032 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/08923444-e4b3-4cdb-8bba-fe70a9b5c218-secret-grpc-tls\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.327187 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.327112 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/08923444-e4b3-4cdb-8bba-fe70a9b5c218-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.327187 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.327165 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/08923444-e4b3-4cdb-8bba-fe70a9b5c218-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.327531 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.327501 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08923444-e4b3-4cdb-8bba-fe70a9b5c218-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.328088 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.328062 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/08923444-e4b3-4cdb-8bba-fe70a9b5c218-secret-thanos-querier-tls\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.328450 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.328429 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08923444-e4b3-4cdb-8bba-fe70a9b5c218-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.333295 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.333264 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc82m\" (UniqueName: \"kubernetes.io/projected/08923444-e4b3-4cdb-8bba-fe70a9b5c218-kube-api-access-cc82m\") pod \"thanos-querier-7d9499477c-2lz6w\" (UID: \"08923444-e4b3-4cdb-8bba-fe70a9b5c218\") " pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.415153 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.415056 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" event={"ID":"39da47d2-4e42-4d22-955e-10adfc404b57","Type":"ContainerStarted","Data":"80360c67745bcd6ebb1dbc7cb1edec96fadc0df42875e2e23d38d523ec8bd689"} Jul 13 19:23:08.416660 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.416561 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57caca18-ea13-445b-a21a-016bb5a0f9c0","Type":"ContainerStarted","Data":"7562ee0c27032504efd0b520be89a667140a35725d1fbc5523df9a6ead863bd5"} Jul 13 19:23:08.418572 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.418528 2556 generic.go:356] "Generic (PLEG): container finished" podID="e4aa6015-fd26-40b5-abfe-d3df70e1c058" containerID="c698c0abc5ef6e8b73d6c16376cbf98d46b5b22a4327b1ca89b0f31e82162249" exitCode=0 Jul 13 19:23:08.418742 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.418619 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qg567" event={"ID":"e4aa6015-fd26-40b5-abfe-d3df70e1c058","Type":"ContainerDied","Data":"c698c0abc5ef6e8b73d6c16376cbf98d46b5b22a4327b1ca89b0f31e82162249"} Jul 13 19:23:08.435525 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.435480 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:08.618589 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:08.618467 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d9499477c-2lz6w"] Jul 13 19:23:09.140437 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:23:09.140401 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08923444_e4b3_4cdb_8bba_fe70a9b5c218.slice/crio-ae2e70af9dc9886daf110a72046128f025489c15e021d93d3d9be7691bb581d8 WatchSource:0}: Error finding container ae2e70af9dc9886daf110a72046128f025489c15e021d93d3d9be7691bb581d8: Status 404 returned error can't find the container with id ae2e70af9dc9886daf110a72046128f025489c15e021d93d3d9be7691bb581d8 Jul 13 19:23:09.427687 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:09.427588 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qg567" event={"ID":"e4aa6015-fd26-40b5-abfe-d3df70e1c058","Type":"ContainerStarted","Data":"88f15d77c9232f9f84ded06ec9abd141de57c5ebb99ef9d634c022ea9733840c"} Jul 13 19:23:09.427687 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:09.427661 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qg567" event={"ID":"e4aa6015-fd26-40b5-abfe-d3df70e1c058","Type":"ContainerStarted","Data":"01c021e84819c331d300c392abe8123c0344c26ecdb6cabd396fc9eba9a0c735"} Jul 13 19:23:09.429596 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:09.429558 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" event={"ID":"08923444-e4b3-4cdb-8bba-fe70a9b5c218","Type":"ContainerStarted","Data":"ae2e70af9dc9886daf110a72046128f025489c15e021d93d3d9be7691bb581d8"} Jul 13 19:23:10.378730 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.378693 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-k24np" Jul 13 19:23:10.395680 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.395610 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qg567" podStartSLOduration=3.27946009 podStartE2EDuration="4.395584623s" totalImagesPullingTime="1.040813735s" totalInitContainerRuntime="75.310798ms" isStatefulPod=true podCreationTimestamp="2026-07-13 19:23:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:23:09.454562598 +0000 UTC m=+84.568470334" watchObservedRunningTime="2026-07-13 19:23:10.395584623 +0000 UTC m=+85.509492315" Jul 13 19:23:10.438615 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.438566 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" event={"ID":"39da47d2-4e42-4d22-955e-10adfc404b57","Type":"ContainerStarted","Data":"bf967a16e446fda6c74b625bf3d32405a426d16c645f4f25a65f88a6e7b19446"} Jul 13 19:23:10.441555 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.441521 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" event={"ID":"c04a5ce2-c2ae-4036-9a7c-519c30553a48","Type":"ContainerStarted","Data":"4be69b53d3d19edb1768d67b68542a8fe91429c622988455e24ba4c9d8bc3113"} Jul 13 19:23:10.441709 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.441593 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" event={"ID":"c04a5ce2-c2ae-4036-9a7c-519c30553a48","Type":"ContainerStarted","Data":"2f513428517d0f72467d523690cf69a026bb1d700f757792753804f168895aa4"} Jul 13 19:23:10.441709 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.441636 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" event={"ID":"c04a5ce2-c2ae-4036-9a7c-519c30553a48","Type":"ContainerStarted","Data":"6677a865a101a246424e46f2f64b67b45ff6a654129baab48283d563045c317e"} Jul 13 19:23:10.443633 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.443607 2556 generic.go:356] "Generic (PLEG): container finished" podID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerID="d8a7ae2f6719662e4d133c8353620813468aa5607ac77c2c370ca42845b97643" exitCode=0 Jul 13 19:23:10.443805 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.443692 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57caca18-ea13-445b-a21a-016bb5a0f9c0","Type":"ContainerDied","Data":"d8a7ae2f6719662e4d133c8353620813468aa5607ac77c2c370ca42845b97643"} Jul 13 19:23:10.454988 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.454957 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7669c847b-whh8v"] Jul 13 19:23:10.457847 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.457820 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.460548 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.460519 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 13 19:23:10.460705 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.460606 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 13 19:23:10.460705 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.460519 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 13 19:23:10.460912 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.460890 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-5uvp49coij0a5\"" Jul 13 19:23:10.461037 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.460959 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 13 19:23:10.461037 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.461012 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qdz2g\"" Jul 13 19:23:10.463728 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.463114 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-2gxgn" podStartSLOduration=2.545659049 podStartE2EDuration="4.463092737s" totalImagesPullingTime="1.917433688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:23:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:23:10.461359805 +0000 UTC m=+85.575267544" watchObservedRunningTime="2026-07-13 19:23:10.463092737 +0000 UTC m=+85.577000451" Jul 13 19:23:10.468940 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.468842 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7669c847b-whh8v"] Jul 13 19:23:10.485156 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.485091 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-c8pz6" podStartSLOduration=1.6124367670000002 podStartE2EDuration="4.485066103s" totalImagesPullingTime="2.872629336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:23:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:23:10.483354508 +0000 UTC m=+85.597262206" watchObservedRunningTime="2026-07-13 19:23:10.485066103 +0000 UTC m=+85.598973816" Jul 13 19:23:10.543798 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.543723 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/334df410-3d09-4551-87e8-4f7184fdba55-secret-metrics-server-tls\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.544377 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.544325 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ns5z\" (UniqueName: \"kubernetes.io/projected/334df410-3d09-4551-87e8-4f7184fdba55-kube-api-access-7ns5z\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.544516 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.544396 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/334df410-3d09-4551-87e8-4f7184fdba55-metrics-server-audit-profiles\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.544516 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.544474 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/334df410-3d09-4551-87e8-4f7184fdba55-secret-metrics-server-client-certs\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.544648 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.544567 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/334df410-3d09-4551-87e8-4f7184fdba55-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.544870 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.544836 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/334df410-3d09-4551-87e8-4f7184fdba55-client-ca-bundle\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.544974 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.544932 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/334df410-3d09-4551-87e8-4f7184fdba55-audit-log\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.646081 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.646036 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ns5z\" (UniqueName: \"kubernetes.io/projected/334df410-3d09-4551-87e8-4f7184fdba55-kube-api-access-7ns5z\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.646081 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.646093 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/334df410-3d09-4551-87e8-4f7184fdba55-metrics-server-audit-profiles\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.646361 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.646135 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/334df410-3d09-4551-87e8-4f7184fdba55-secret-metrics-server-client-certs\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.646361 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.646167 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/334df410-3d09-4551-87e8-4f7184fdba55-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.646361 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.646229 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/334df410-3d09-4551-87e8-4f7184fdba55-client-ca-bundle\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.646361 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.646266 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/334df410-3d09-4551-87e8-4f7184fdba55-audit-log\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.646361 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.646299 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/334df410-3d09-4551-87e8-4f7184fdba55-secret-metrics-server-tls\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.646863 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.646810 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/334df410-3d09-4551-87e8-4f7184fdba55-audit-log\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.647116 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.647063 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/334df410-3d09-4551-87e8-4f7184fdba55-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.647345 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.647319 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/334df410-3d09-4551-87e8-4f7184fdba55-metrics-server-audit-profiles\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.649451 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.649424 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/334df410-3d09-4551-87e8-4f7184fdba55-secret-metrics-server-client-certs\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.649623 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.649598 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/334df410-3d09-4551-87e8-4f7184fdba55-secret-metrics-server-tls\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.649758 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.649595 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/334df410-3d09-4551-87e8-4f7184fdba55-client-ca-bundle\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.655649 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.655622 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ns5z\" (UniqueName: \"kubernetes.io/projected/334df410-3d09-4551-87e8-4f7184fdba55-kube-api-access-7ns5z\") pod \"metrics-server-7669c847b-whh8v\" (UID: \"334df410-3d09-4551-87e8-4f7184fdba55\") " pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.772006 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.771911 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:10.832847 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.832805 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-pnb99"] Jul 13 19:23:10.835912 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.835883 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-pnb99" Jul 13 19:23:10.838015 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.837988 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 13 19:23:10.838158 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.838062 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-kwdcn\"" Jul 13 19:23:10.855627 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.855598 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-pnb99"] Jul 13 19:23:10.950059 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.950019 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbdtv\" (UniqueName: \"kubernetes.io/projected/a7bdc7c5-be2c-4135-a6d2-1f901510cf6d-kube-api-access-cbdtv\") pod \"monitoring-plugin-5f68df7f58-pnb99\" (UID: \"a7bdc7c5-be2c-4135-a6d2-1f901510cf6d\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-pnb99" Jul 13 19:23:10.950233 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:10.950068 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a7bdc7c5-be2c-4135-a6d2-1f901510cf6d-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-pnb99\" (UID: \"a7bdc7c5-be2c-4135-a6d2-1f901510cf6d\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-pnb99" Jul 13 19:23:11.006255 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:11.006134 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7669c847b-whh8v"] Jul 13 19:23:11.011237 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:23:11.011207 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod334df410_3d09_4551_87e8_4f7184fdba55.slice/crio-32b54e035c92974040a28dc9a9daf229e482a634e0468c1b907f01fc7c5ca987 WatchSource:0}: Error finding container 32b54e035c92974040a28dc9a9daf229e482a634e0468c1b907f01fc7c5ca987: Status 404 returned error can't find the container with id 32b54e035c92974040a28dc9a9daf229e482a634e0468c1b907f01fc7c5ca987 Jul 13 19:23:11.051189 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:11.051150 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbdtv\" (UniqueName: \"kubernetes.io/projected/a7bdc7c5-be2c-4135-a6d2-1f901510cf6d-kube-api-access-cbdtv\") pod \"monitoring-plugin-5f68df7f58-pnb99\" (UID: \"a7bdc7c5-be2c-4135-a6d2-1f901510cf6d\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-pnb99" Jul 13 19:23:11.051331 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:11.051207 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a7bdc7c5-be2c-4135-a6d2-1f901510cf6d-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-pnb99\" (UID: \"a7bdc7c5-be2c-4135-a6d2-1f901510cf6d\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-pnb99" Jul 13 19:23:11.054191 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:11.054156 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a7bdc7c5-be2c-4135-a6d2-1f901510cf6d-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-pnb99\" (UID: \"a7bdc7c5-be2c-4135-a6d2-1f901510cf6d\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-pnb99" Jul 13 19:23:11.064066 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:11.064000 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbdtv\" (UniqueName: \"kubernetes.io/projected/a7bdc7c5-be2c-4135-a6d2-1f901510cf6d-kube-api-access-cbdtv\") pod \"monitoring-plugin-5f68df7f58-pnb99\" (UID: \"a7bdc7c5-be2c-4135-a6d2-1f901510cf6d\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-pnb99" Jul 13 19:23:11.153438 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:11.153401 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-pnb99" Jul 13 19:23:11.186728 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:11.186684 2556 patch_prober.go:28] interesting pod/image-registry-5b6d67588f-zwg7r container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 13 19:23:11.186921 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:11.186755 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" podUID="af3abc1b-2644-468b-9144-d199a1417e34" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 19:23:11.307803 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:11.307749 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-pnb99"] Jul 13 19:23:11.310518 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:23:11.310432 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7bdc7c5_be2c_4135_a6d2_1f901510cf6d.slice/crio-3bab9f8d4dbf69c655b4a1395a5e52b1d8bbb37e9a9afcbfd59dc71381c9bd3a WatchSource:0}: Error finding container 3bab9f8d4dbf69c655b4a1395a5e52b1d8bbb37e9a9afcbfd59dc71381c9bd3a: Status 404 returned error can't find the container with id 3bab9f8d4dbf69c655b4a1395a5e52b1d8bbb37e9a9afcbfd59dc71381c9bd3a Jul 13 19:23:11.450726 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:11.450691 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" event={"ID":"08923444-e4b3-4cdb-8bba-fe70a9b5c218","Type":"ContainerStarted","Data":"8796944c234dc3605a538d1d6993ed29ad637e0de4a0b06d001714d731977b22"} Jul 13 19:23:11.451179 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:11.450762 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" event={"ID":"08923444-e4b3-4cdb-8bba-fe70a9b5c218","Type":"ContainerStarted","Data":"6d8743c450978cfb5b7e79cb3ebf06a3fedc105d6c7737c5adf4aa2f980102a5"} Jul 13 19:23:11.451179 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:11.450818 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" event={"ID":"08923444-e4b3-4cdb-8bba-fe70a9b5c218","Type":"ContainerStarted","Data":"ae4723419c9f39b86d84177dfa37a8512144ff7c6e08aa12d2354ea145c35f95"} Jul 13 19:23:11.451993 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:11.451963 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-pnb99" event={"ID":"a7bdc7c5-be2c-4135-a6d2-1f901510cf6d","Type":"ContainerStarted","Data":"3bab9f8d4dbf69c655b4a1395a5e52b1d8bbb37e9a9afcbfd59dc71381c9bd3a"} Jul 13 19:23:11.453137 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:11.453113 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7669c847b-whh8v" event={"ID":"334df410-3d09-4551-87e8-4f7184fdba55","Type":"ContainerStarted","Data":"32b54e035c92974040a28dc9a9daf229e482a634e0468c1b907f01fc7c5ca987"} Jul 13 19:23:12.466703 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.466376 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57caca18-ea13-445b-a21a-016bb5a0f9c0","Type":"ContainerStarted","Data":"d80571a0b2bcba93e5bd22284dc53272da39a0c2c12ccbfdb756c1f45caf373b"} Jul 13 19:23:12.466703 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.466469 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57caca18-ea13-445b-a21a-016bb5a0f9c0","Type":"ContainerStarted","Data":"fd35d134c7d35d868c3480c863f41e248a128172725c6422a0d67f6bcb990897"} Jul 13 19:23:12.466703 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.466508 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57caca18-ea13-445b-a21a-016bb5a0f9c0","Type":"ContainerStarted","Data":"432eab62005f7b54d460e4f1a5048aed82fe3a73c882e8266deadadc7e6e7695"} Jul 13 19:23:12.466703 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.466545 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57caca18-ea13-445b-a21a-016bb5a0f9c0","Type":"ContainerStarted","Data":"6304ed5b554dc01412e563cfd49a27fe2c586cb30410ab27297edaa75ef42665"} Jul 13 19:23:12.466703 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.466581 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57caca18-ea13-445b-a21a-016bb5a0f9c0","Type":"ContainerStarted","Data":"8183fdd1f287bcafcd419402e098a192cf614834b7c230bc83a84f166b1d240b"} Jul 13 19:23:12.708544 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.708378 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c6857b649-xft4v"] Jul 13 19:23:12.711299 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.711272 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:12.714477 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.714247 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 19:23:12.714477 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.714273 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 19:23:12.715241 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.715173 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-svwt5\"" Jul 13 19:23:12.715524 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.715471 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 13 19:23:12.715635 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.715616 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 13 19:23:12.715703 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.715638 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 19:23:12.716321 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.715764 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 19:23:12.716321 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.716133 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 19:23:12.726035 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.725425 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c6857b649-xft4v"] Jul 13 19:23:12.775482 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.775443 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36b77791-00e3-420e-8939-02e9fa1168ba-console-oauth-config\") pod \"console-5c6857b649-xft4v\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:12.775675 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.775497 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36b77791-00e3-420e-8939-02e9fa1168ba-oauth-serving-cert\") pod \"console-5c6857b649-xft4v\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:12.775675 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.775592 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36b77791-00e3-420e-8939-02e9fa1168ba-console-serving-cert\") pod \"console-5c6857b649-xft4v\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:12.775675 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.775627 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36b77791-00e3-420e-8939-02e9fa1168ba-console-config\") pod \"console-5c6857b649-xft4v\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:12.775675 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.775648 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36b77791-00e3-420e-8939-02e9fa1168ba-service-ca\") pod \"console-5c6857b649-xft4v\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:12.775675 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.775668 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8kqv\" (UniqueName: \"kubernetes.io/projected/36b77791-00e3-420e-8939-02e9fa1168ba-kube-api-access-x8kqv\") pod \"console-5c6857b649-xft4v\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:12.877500 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.877455 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36b77791-00e3-420e-8939-02e9fa1168ba-console-oauth-config\") pod \"console-5c6857b649-xft4v\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:12.877703 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.877510 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36b77791-00e3-420e-8939-02e9fa1168ba-oauth-serving-cert\") pod \"console-5c6857b649-xft4v\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:12.877703 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.877602 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36b77791-00e3-420e-8939-02e9fa1168ba-console-serving-cert\") pod \"console-5c6857b649-xft4v\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:12.877703 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.877634 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36b77791-00e3-420e-8939-02e9fa1168ba-console-config\") pod \"console-5c6857b649-xft4v\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:12.877703 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.877658 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36b77791-00e3-420e-8939-02e9fa1168ba-service-ca\") pod \"console-5c6857b649-xft4v\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:12.878012 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.877706 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8kqv\" (UniqueName: \"kubernetes.io/projected/36b77791-00e3-420e-8939-02e9fa1168ba-kube-api-access-x8kqv\") pod \"console-5c6857b649-xft4v\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:12.878488 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.878457 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36b77791-00e3-420e-8939-02e9fa1168ba-service-ca\") pod \"console-5c6857b649-xft4v\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:12.878585 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.878546 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36b77791-00e3-420e-8939-02e9fa1168ba-console-config\") pod \"console-5c6857b649-xft4v\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:12.879125 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.879093 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36b77791-00e3-420e-8939-02e9fa1168ba-oauth-serving-cert\") pod \"console-5c6857b649-xft4v\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:12.881861 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.881763 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36b77791-00e3-420e-8939-02e9fa1168ba-console-oauth-config\") pod \"console-5c6857b649-xft4v\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:12.882041 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.882012 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36b77791-00e3-420e-8939-02e9fa1168ba-console-serving-cert\") pod \"console-5c6857b649-xft4v\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:12.891306 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:12.891234 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8kqv\" (UniqueName: \"kubernetes.io/projected/36b77791-00e3-420e-8939-02e9fa1168ba-kube-api-access-x8kqv\") pod \"console-5c6857b649-xft4v\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:13.029341 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:13.029233 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:13.692056 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:13.692010 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c6857b649-xft4v"] Jul 13 19:23:13.695160 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:23:13.695121 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36b77791_00e3_420e_8939_02e9fa1168ba.slice/crio-ccd39166f5b72d8b4b332798bc7dd9395704827d3afa51076855a3af34e8edcf WatchSource:0}: Error finding container ccd39166f5b72d8b4b332798bc7dd9395704827d3afa51076855a3af34e8edcf: Status 404 returned error can't find the container with id ccd39166f5b72d8b4b332798bc7dd9395704827d3afa51076855a3af34e8edcf Jul 13 19:23:14.342167 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:14.342120 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b6d67588f-zwg7r" Jul 13 19:23:14.479984 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:14.479886 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57caca18-ea13-445b-a21a-016bb5a0f9c0","Type":"ContainerStarted","Data":"9f8b087a6e49cd425e2c4f4fcb01b40577984245f478b11b74e676c1f10d6c26"} Jul 13 19:23:14.481629 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:14.481577 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c6857b649-xft4v" event={"ID":"36b77791-00e3-420e-8939-02e9fa1168ba","Type":"ContainerStarted","Data":"ccd39166f5b72d8b4b332798bc7dd9395704827d3afa51076855a3af34e8edcf"} Jul 13 19:23:14.485174 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:14.485067 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" event={"ID":"08923444-e4b3-4cdb-8bba-fe70a9b5c218","Type":"ContainerStarted","Data":"9cbe2e7cbb6bc87d7efecb77000c19d1bd42656f95c66158533b5f3d30341bdf"} Jul 13 19:23:14.485174 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:14.485144 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" event={"ID":"08923444-e4b3-4cdb-8bba-fe70a9b5c218","Type":"ContainerStarted","Data":"233f3f6da76b50191542210c2841bda6f57f92d155e1dcb45c5dac015602f6e9"} Jul 13 19:23:14.485379 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:14.485191 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" event={"ID":"08923444-e4b3-4cdb-8bba-fe70a9b5c218","Type":"ContainerStarted","Data":"c936bbc389d200c7799167927b43b35d984ba5100bcdc40ee90ad9b997fe967f"} Jul 13 19:23:14.485379 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:14.485325 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:14.487578 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:14.487153 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-pnb99" event={"ID":"a7bdc7c5-be2c-4135-a6d2-1f901510cf6d","Type":"ContainerStarted","Data":"3f93b24116102b6316c182d16af1d14e90da285f4fb6a25d68e44c118f43c161"} Jul 13 19:23:14.487578 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:14.487292 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-pnb99" Jul 13 19:23:14.489037 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:14.488985 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7669c847b-whh8v" event={"ID":"334df410-3d09-4551-87e8-4f7184fdba55","Type":"ContainerStarted","Data":"9aa28ea06ca3d2ca32c62d4a70a65d2928772f34d072b16b9e5f22cd5c55803f"} Jul 13 19:23:14.493559 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:14.493534 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-pnb99" Jul 13 19:23:14.510454 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:14.510383 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.961117275 podStartE2EDuration="7.510352808s" totalImagesPullingTime="4.525519454s" totalInitContainerRuntime="23.716079ms" isStatefulPod=false podCreationTimestamp="2026-07-13 19:23:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:23:14.509515457 +0000 UTC m=+89.623423167" watchObservedRunningTime="2026-07-13 19:23:14.510352808 +0000 UTC m=+89.624260518" Jul 13 19:23:14.528880 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:14.528823 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-pnb99" podStartSLOduration=2.310833186 podStartE2EDuration="4.528799334s" totalImagesPullingTime="2.217966148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:23:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:23:14.528023119 +0000 UTC m=+89.641930828" watchObservedRunningTime="2026-07-13 19:23:14.528799334 +0000 UTC m=+89.642707042" Jul 13 19:23:14.555093 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:14.555017 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" podStartSLOduration=2.411071962 podStartE2EDuration="6.55499431s" totalImagesPullingTime="4.143922348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:23:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:23:14.55314311 +0000 UTC m=+89.667050843" watchObservedRunningTime="2026-07-13 19:23:14.55499431 +0000 UTC m=+89.668902020" Jul 13 19:23:14.572268 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:14.572203 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7669c847b-whh8v" podStartSLOduration=2.056081602 podStartE2EDuration="4.57217898s" totalImagesPullingTime="2.516097378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:23:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:23:14.571969696 +0000 UTC m=+89.685877419" watchObservedRunningTime="2026-07-13 19:23:14.57217898 +0000 UTC m=+89.686086690" Jul 13 19:23:17.506830 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:17.506763 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c6857b649-xft4v" event={"ID":"36b77791-00e3-420e-8939-02e9fa1168ba","Type":"ContainerStarted","Data":"8b31dae0966867d417dd35301b7b5aedc5456d6724e4ea36cbae8fd04dc504ae"} Jul 13 19:23:17.531206 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:17.531138 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c6857b649-xft4v" podStartSLOduration=2.385388541 podStartE2EDuration="5.531115378s" totalImagesPullingTime="3.145726837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:23:17.529452222 +0000 UTC m=+92.643359950" watchObservedRunningTime="2026-07-13 19:23:17.531115378 +0000 UTC m=+92.645023088" Jul 13 19:23:20.485558 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.485517 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f87c7ffb-44kt8"] Jul 13 19:23:20.488302 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.488281 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.497657 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.497631 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 19:23:20.501293 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.501268 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7d9499477c-2lz6w" Jul 13 19:23:20.503556 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.503528 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f87c7ffb-44kt8"] Jul 13 19:23:20.648563 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.648503 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjfs4\" (UniqueName: \"kubernetes.io/projected/969f35ca-651b-4a65-a6fb-51a94419bb8f-kube-api-access-rjfs4\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.648800 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.648694 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/969f35ca-651b-4a65-a6fb-51a94419bb8f-console-oauth-config\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.648800 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.648728 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-trusted-ca-bundle\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.648945 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.648879 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-service-ca\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.649075 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.649040 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/969f35ca-651b-4a65-a6fb-51a94419bb8f-console-serving-cert\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.649125 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.649076 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-console-config\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.649125 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.649103 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-oauth-serving-cert\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.749976 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.749871 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/969f35ca-651b-4a65-a6fb-51a94419bb8f-console-serving-cert\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.749976 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.749923 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-console-config\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.750203 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.750031 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-oauth-serving-cert\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.750203 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.750080 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjfs4\" (UniqueName: \"kubernetes.io/projected/969f35ca-651b-4a65-a6fb-51a94419bb8f-kube-api-access-rjfs4\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.750203 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.750130 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/969f35ca-651b-4a65-a6fb-51a94419bb8f-console-oauth-config\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.750203 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.750146 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-trusted-ca-bundle\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.750203 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.750181 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-service-ca\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.750753 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.750710 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-console-config\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.750753 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.750709 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-oauth-serving-cert\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.750974 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.750953 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-service-ca\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.751047 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.751031 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-trusted-ca-bundle\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.752533 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.752514 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/969f35ca-651b-4a65-a6fb-51a94419bb8f-console-serving-cert\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.752587 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.752540 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/969f35ca-651b-4a65-a6fb-51a94419bb8f-console-oauth-config\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.758154 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.758126 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjfs4\" (UniqueName: \"kubernetes.io/projected/969f35ca-651b-4a65-a6fb-51a94419bb8f-kube-api-access-rjfs4\") pod \"console-7f87c7ffb-44kt8\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.798398 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.798353 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:20.937254 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:20.937226 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f87c7ffb-44kt8"] Jul 13 19:23:20.939885 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:23:20.939840 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod969f35ca_651b_4a65_a6fb_51a94419bb8f.slice/crio-48a1671c822b2c9afdfa51729d23888b07e6898e881f1a36ce8b9310834d2a12 WatchSource:0}: Error finding container 48a1671c822b2c9afdfa51729d23888b07e6898e881f1a36ce8b9310834d2a12: Status 404 returned error can't find the container with id 48a1671c822b2c9afdfa51729d23888b07e6898e881f1a36ce8b9310834d2a12 Jul 13 19:23:21.526970 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:21.526928 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f87c7ffb-44kt8" event={"ID":"969f35ca-651b-4a65-a6fb-51a94419bb8f","Type":"ContainerStarted","Data":"47a04ac1dbd71409c3bba905ca1df031cd36212eec93f20936aa294fb724d866"} Jul 13 19:23:21.527612 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:21.527585 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f87c7ffb-44kt8" event={"ID":"969f35ca-651b-4a65-a6fb-51a94419bb8f","Type":"ContainerStarted","Data":"48a1671c822b2c9afdfa51729d23888b07e6898e881f1a36ce8b9310834d2a12"} Jul 13 19:23:21.546931 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:21.546863 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f87c7ffb-44kt8" podStartSLOduration=1.5468403240000002 podStartE2EDuration="1.546840324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:23:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:23:21.545529057 +0000 UTC m=+96.659436793" watchObservedRunningTime="2026-07-13 19:23:21.546840324 +0000 UTC m=+96.660748033" Jul 13 19:23:23.029836 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:23.029766 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:23.030337 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:23.029865 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:23.035658 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:23.035632 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:23.538491 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:23.538461 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:29.363936 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:29.363900 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-f2lcv" Jul 13 19:23:30.772231 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:30.772189 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:30.772639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:30.772287 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:30.798798 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:30.798735 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:30.798945 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:30.798825 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:30.803801 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:30.803759 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:31.573251 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:31.573219 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:23:31.631512 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:31.631477 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c6857b649-xft4v"] Jul 13 19:23:50.777579 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:50.777544 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:50.781504 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:50.781476 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7669c847b-whh8v" Jul 13 19:23:56.661134 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:56.661062 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c6857b649-xft4v" podUID="36b77791-00e3-420e-8939-02e9fa1168ba" containerName="console" containerID="cri-o://8b31dae0966867d417dd35301b7b5aedc5456d6724e4ea36cbae8fd04dc504ae" gracePeriod=15 Jul 13 19:23:56.910658 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:56.910631 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:56.962760 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:56.962661 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36b77791-00e3-420e-8939-02e9fa1168ba-oauth-serving-cert\") pod \"36b77791-00e3-420e-8939-02e9fa1168ba\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " Jul 13 19:23:56.962760 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:56.962701 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36b77791-00e3-420e-8939-02e9fa1168ba-service-ca\") pod \"36b77791-00e3-420e-8939-02e9fa1168ba\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " Jul 13 19:23:56.962760 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:56.962723 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8kqv\" (UniqueName: \"kubernetes.io/projected/36b77791-00e3-420e-8939-02e9fa1168ba-kube-api-access-x8kqv\") pod \"36b77791-00e3-420e-8939-02e9fa1168ba\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " Jul 13 19:23:56.962760 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:56.962740 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36b77791-00e3-420e-8939-02e9fa1168ba-console-config\") pod \"36b77791-00e3-420e-8939-02e9fa1168ba\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " Jul 13 19:23:56.963081 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:56.962813 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36b77791-00e3-420e-8939-02e9fa1168ba-console-oauth-config\") pod \"36b77791-00e3-420e-8939-02e9fa1168ba\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " Jul 13 19:23:56.963081 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:56.962848 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36b77791-00e3-420e-8939-02e9fa1168ba-console-serving-cert\") pod \"36b77791-00e3-420e-8939-02e9fa1168ba\" (UID: \"36b77791-00e3-420e-8939-02e9fa1168ba\") " Jul 13 19:23:56.963243 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:56.963216 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36b77791-00e3-420e-8939-02e9fa1168ba-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "36b77791-00e3-420e-8939-02e9fa1168ba" (UID: "36b77791-00e3-420e-8939-02e9fa1168ba"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:23:56.963314 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:56.963224 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36b77791-00e3-420e-8939-02e9fa1168ba-console-config" (OuterVolumeSpecName: "console-config") pod "36b77791-00e3-420e-8939-02e9fa1168ba" (UID: "36b77791-00e3-420e-8939-02e9fa1168ba"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:23:56.963314 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:56.963285 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36b77791-00e3-420e-8939-02e9fa1168ba-service-ca" (OuterVolumeSpecName: "service-ca") pod "36b77791-00e3-420e-8939-02e9fa1168ba" (UID: "36b77791-00e3-420e-8939-02e9fa1168ba"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:23:56.963506 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:56.963487 2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36b77791-00e3-420e-8939-02e9fa1168ba-oauth-serving-cert\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:23:56.963585 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:56.963511 2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36b77791-00e3-420e-8939-02e9fa1168ba-service-ca\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:23:56.963585 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:56.963543 2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36b77791-00e3-420e-8939-02e9fa1168ba-console-config\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:23:56.965315 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:56.965288 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36b77791-00e3-420e-8939-02e9fa1168ba-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "36b77791-00e3-420e-8939-02e9fa1168ba" (UID: "36b77791-00e3-420e-8939-02e9fa1168ba"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:23:56.965416 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:56.965315 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36b77791-00e3-420e-8939-02e9fa1168ba-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "36b77791-00e3-420e-8939-02e9fa1168ba" (UID: "36b77791-00e3-420e-8939-02e9fa1168ba"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:23:56.965416 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:56.965396 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36b77791-00e3-420e-8939-02e9fa1168ba-kube-api-access-x8kqv" (OuterVolumeSpecName: "kube-api-access-x8kqv") pod "36b77791-00e3-420e-8939-02e9fa1168ba" (UID: "36b77791-00e3-420e-8939-02e9fa1168ba"). InnerVolumeSpecName "kube-api-access-x8kqv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:23:57.064426 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:57.064370 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8kqv\" (UniqueName: \"kubernetes.io/projected/36b77791-00e3-420e-8939-02e9fa1168ba-kube-api-access-x8kqv\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:23:57.064426 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:57.064419 2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36b77791-00e3-420e-8939-02e9fa1168ba-console-oauth-config\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:23:57.064426 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:57.064429 2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36b77791-00e3-420e-8939-02e9fa1168ba-console-serving-cert\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:23:57.689986 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:57.689949 2556 generic.go:356] "Generic (PLEG): container finished" podID="36b77791-00e3-420e-8939-02e9fa1168ba" containerID="8b31dae0966867d417dd35301b7b5aedc5456d6724e4ea36cbae8fd04dc504ae" exitCode=0 Jul 13 19:23:57.690501 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:57.690001 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c6857b649-xft4v" Jul 13 19:23:57.690501 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:57.690031 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c6857b649-xft4v" event={"ID":"36b77791-00e3-420e-8939-02e9fa1168ba","Type":"ContainerDied","Data":"8b31dae0966867d417dd35301b7b5aedc5456d6724e4ea36cbae8fd04dc504ae"} Jul 13 19:23:57.690501 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:57.690085 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c6857b649-xft4v" event={"ID":"36b77791-00e3-420e-8939-02e9fa1168ba","Type":"ContainerDied","Data":"ccd39166f5b72d8b4b332798bc7dd9395704827d3afa51076855a3af34e8edcf"} Jul 13 19:23:57.690501 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:57.690106 2556 scope.go:117] "RemoveContainer" containerID="8b31dae0966867d417dd35301b7b5aedc5456d6724e4ea36cbae8fd04dc504ae" Jul 13 19:23:57.699245 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:57.699221 2556 scope.go:117] "RemoveContainer" containerID="8b31dae0966867d417dd35301b7b5aedc5456d6724e4ea36cbae8fd04dc504ae" Jul 13 19:23:57.699573 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:23:57.699551 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b31dae0966867d417dd35301b7b5aedc5456d6724e4ea36cbae8fd04dc504ae\": container with ID starting with 8b31dae0966867d417dd35301b7b5aedc5456d6724e4ea36cbae8fd04dc504ae not found: ID does not exist" containerID="8b31dae0966867d417dd35301b7b5aedc5456d6724e4ea36cbae8fd04dc504ae" Jul 13 19:23:57.699649 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:57.699583 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b31dae0966867d417dd35301b7b5aedc5456d6724e4ea36cbae8fd04dc504ae"} err="failed to get container status \"8b31dae0966867d417dd35301b7b5aedc5456d6724e4ea36cbae8fd04dc504ae\": rpc error: code = NotFound desc = could not find container \"8b31dae0966867d417dd35301b7b5aedc5456d6724e4ea36cbae8fd04dc504ae\": container with ID starting with 8b31dae0966867d417dd35301b7b5aedc5456d6724e4ea36cbae8fd04dc504ae not found: ID does not exist" Jul 13 19:23:57.716418 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:57.716382 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c6857b649-xft4v"] Jul 13 19:23:57.719793 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:57.719743 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c6857b649-xft4v"] Jul 13 19:23:58.848515 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:23:58.848474 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36b77791-00e3-420e-8939-02e9fa1168ba" path="/var/lib/kubelet/pods/36b77791-00e3-420e-8939-02e9fa1168ba/volumes" Jul 13 19:24:26.397040 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:26.397000 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 19:24:26.397486 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:26.397404 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="alertmanager" containerID="cri-o://8183fdd1f287bcafcd419402e098a192cf614834b7c230bc83a84f166b1d240b" gracePeriod=120 Jul 13 19:24:26.397567 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:26.397463 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="kube-rbac-proxy-metric" containerID="cri-o://d80571a0b2bcba93e5bd22284dc53272da39a0c2c12ccbfdb756c1f45caf373b" gracePeriod=120 Jul 13 19:24:26.397567 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:26.397510 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="config-reloader" containerID="cri-o://6304ed5b554dc01412e563cfd49a27fe2c586cb30410ab27297edaa75ef42665" gracePeriod=120 Jul 13 19:24:26.397567 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:26.397509 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="kube-rbac-proxy-web" containerID="cri-o://432eab62005f7b54d460e4f1a5048aed82fe3a73c882e8266deadadc7e6e7695" gracePeriod=120 Jul 13 19:24:26.397729 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:26.397608 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="prom-label-proxy" containerID="cri-o://9f8b087a6e49cd425e2c4f4fcb01b40577984245f478b11b74e676c1f10d6c26" gracePeriod=120 Jul 13 19:24:26.397729 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:26.397536 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="kube-rbac-proxy" containerID="cri-o://fd35d134c7d35d868c3480c863f41e248a128172725c6422a0d67f6bcb990897" gracePeriod=120 Jul 13 19:24:26.812863 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:26.812826 2556 generic.go:356] "Generic (PLEG): container finished" podID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerID="9f8b087a6e49cd425e2c4f4fcb01b40577984245f478b11b74e676c1f10d6c26" exitCode=0 Jul 13 19:24:26.812863 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:26.812852 2556 generic.go:356] "Generic (PLEG): container finished" podID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerID="d80571a0b2bcba93e5bd22284dc53272da39a0c2c12ccbfdb756c1f45caf373b" exitCode=0 Jul 13 19:24:26.812863 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:26.812859 2556 generic.go:356] "Generic (PLEG): container finished" podID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerID="fd35d134c7d35d868c3480c863f41e248a128172725c6422a0d67f6bcb990897" exitCode=0 Jul 13 19:24:26.812863 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:26.812865 2556 generic.go:356] "Generic (PLEG): container finished" podID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerID="6304ed5b554dc01412e563cfd49a27fe2c586cb30410ab27297edaa75ef42665" exitCode=0 Jul 13 19:24:26.812863 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:26.812872 2556 generic.go:356] "Generic (PLEG): container finished" podID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerID="8183fdd1f287bcafcd419402e098a192cf614834b7c230bc83a84f166b1d240b" exitCode=0 Jul 13 19:24:26.813167 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:26.812903 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57caca18-ea13-445b-a21a-016bb5a0f9c0","Type":"ContainerDied","Data":"9f8b087a6e49cd425e2c4f4fcb01b40577984245f478b11b74e676c1f10d6c26"} Jul 13 19:24:26.813167 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:26.812973 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57caca18-ea13-445b-a21a-016bb5a0f9c0","Type":"ContainerDied","Data":"d80571a0b2bcba93e5bd22284dc53272da39a0c2c12ccbfdb756c1f45caf373b"} Jul 13 19:24:26.813167 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:26.812999 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57caca18-ea13-445b-a21a-016bb5a0f9c0","Type":"ContainerDied","Data":"fd35d134c7d35d868c3480c863f41e248a128172725c6422a0d67f6bcb990897"} Jul 13 19:24:26.813167 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:26.813025 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57caca18-ea13-445b-a21a-016bb5a0f9c0","Type":"ContainerDied","Data":"6304ed5b554dc01412e563cfd49a27fe2c586cb30410ab27297edaa75ef42665"} Jul 13 19:24:26.813167 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:26.813050 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57caca18-ea13-445b-a21a-016bb5a0f9c0","Type":"ContainerDied","Data":"8183fdd1f287bcafcd419402e098a192cf614834b7c230bc83a84f166b1d240b"} Jul 13 19:24:27.649833 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.649806 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:27.731358 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.731266 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57caca18-ea13-445b-a21a-016bb5a0f9c0-alertmanager-trusted-ca-bundle\") pod \"57caca18-ea13-445b-a21a-016bb5a0f9c0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " Jul 13 19:24:27.731358 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.731303 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-cluster-tls-config\") pod \"57caca18-ea13-445b-a21a-016bb5a0f9c0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " Jul 13 19:24:27.731358 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.731334 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-kube-rbac-proxy\") pod \"57caca18-ea13-445b-a21a-016bb5a0f9c0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " Jul 13 19:24:27.731358 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.731357 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57caca18-ea13-445b-a21a-016bb5a0f9c0-alertmanager-main-db\") pod \"57caca18-ea13-445b-a21a-016bb5a0f9c0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " Jul 13 19:24:27.731693 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.731387 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57caca18-ea13-445b-a21a-016bb5a0f9c0-metrics-client-ca\") pod \"57caca18-ea13-445b-a21a-016bb5a0f9c0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " Jul 13 19:24:27.731693 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.731440 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57caca18-ea13-445b-a21a-016bb5a0f9c0-config-out\") pod \"57caca18-ea13-445b-a21a-016bb5a0f9c0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " Jul 13 19:24:27.731693 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.731471 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"57caca18-ea13-445b-a21a-016bb5a0f9c0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " Jul 13 19:24:27.731693 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.731517 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6qxt\" (UniqueName: \"kubernetes.io/projected/57caca18-ea13-445b-a21a-016bb5a0f9c0-kube-api-access-v6qxt\") pod \"57caca18-ea13-445b-a21a-016bb5a0f9c0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " Jul 13 19:24:27.731693 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.731543 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57caca18-ea13-445b-a21a-016bb5a0f9c0-tls-assets\") pod \"57caca18-ea13-445b-a21a-016bb5a0f9c0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " Jul 13 19:24:27.731693 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.731570 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-web-config\") pod \"57caca18-ea13-445b-a21a-016bb5a0f9c0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " Jul 13 19:24:27.731693 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.731593 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-config-volume\") pod \"57caca18-ea13-445b-a21a-016bb5a0f9c0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " Jul 13 19:24:27.731693 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.731631 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-main-tls\") pod \"57caca18-ea13-445b-a21a-016bb5a0f9c0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " Jul 13 19:24:27.731693 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.731655 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-kube-rbac-proxy-web\") pod \"57caca18-ea13-445b-a21a-016bb5a0f9c0\" (UID: \"57caca18-ea13-445b-a21a-016bb5a0f9c0\") " Jul 13 19:24:27.732162 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.731701 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57caca18-ea13-445b-a21a-016bb5a0f9c0-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "57caca18-ea13-445b-a21a-016bb5a0f9c0" (UID: "57caca18-ea13-445b-a21a-016bb5a0f9c0"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:24:27.732162 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.731740 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57caca18-ea13-445b-a21a-016bb5a0f9c0-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "57caca18-ea13-445b-a21a-016bb5a0f9c0" (UID: "57caca18-ea13-445b-a21a-016bb5a0f9c0"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:24:27.732162 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.731846 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57caca18-ea13-445b-a21a-016bb5a0f9c0-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "57caca18-ea13-445b-a21a-016bb5a0f9c0" (UID: "57caca18-ea13-445b-a21a-016bb5a0f9c0"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:24:27.732162 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.732005 2556 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57caca18-ea13-445b-a21a-016bb5a0f9c0-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:24:27.732162 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.732027 2556 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57caca18-ea13-445b-a21a-016bb5a0f9c0-alertmanager-main-db\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:24:27.732162 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.732044 2556 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57caca18-ea13-445b-a21a-016bb5a0f9c0-metrics-client-ca\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:24:27.734502 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.734473 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57caca18-ea13-445b-a21a-016bb5a0f9c0-kube-api-access-v6qxt" (OuterVolumeSpecName: "kube-api-access-v6qxt") pod "57caca18-ea13-445b-a21a-016bb5a0f9c0" (UID: "57caca18-ea13-445b-a21a-016bb5a0f9c0"). InnerVolumeSpecName "kube-api-access-v6qxt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:24:27.735044 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.734993 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "57caca18-ea13-445b-a21a-016bb5a0f9c0" (UID: "57caca18-ea13-445b-a21a-016bb5a0f9c0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:24:27.735044 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.735006 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57caca18-ea13-445b-a21a-016bb5a0f9c0-config-out" (OuterVolumeSpecName: "config-out") pod "57caca18-ea13-445b-a21a-016bb5a0f9c0" (UID: "57caca18-ea13-445b-a21a-016bb5a0f9c0"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:24:27.735044 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.734999 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57caca18-ea13-445b-a21a-016bb5a0f9c0-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "57caca18-ea13-445b-a21a-016bb5a0f9c0" (UID: "57caca18-ea13-445b-a21a-016bb5a0f9c0"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:24:27.735326 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.735283 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "57caca18-ea13-445b-a21a-016bb5a0f9c0" (UID: "57caca18-ea13-445b-a21a-016bb5a0f9c0"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:24:27.735455 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.735365 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "57caca18-ea13-445b-a21a-016bb5a0f9c0" (UID: "57caca18-ea13-445b-a21a-016bb5a0f9c0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:24:27.735565 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.735550 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-config-volume" (OuterVolumeSpecName: "config-volume") pod "57caca18-ea13-445b-a21a-016bb5a0f9c0" (UID: "57caca18-ea13-445b-a21a-016bb5a0f9c0"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:24:27.735755 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.735740 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "57caca18-ea13-445b-a21a-016bb5a0f9c0" (UID: "57caca18-ea13-445b-a21a-016bb5a0f9c0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:24:27.739398 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.739259 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "57caca18-ea13-445b-a21a-016bb5a0f9c0" (UID: "57caca18-ea13-445b-a21a-016bb5a0f9c0"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:24:27.746658 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.746629 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-web-config" (OuterVolumeSpecName: "web-config") pod "57caca18-ea13-445b-a21a-016bb5a0f9c0" (UID: "57caca18-ea13-445b-a21a-016bb5a0f9c0"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:24:27.820829 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.820794 2556 generic.go:356] "Generic (PLEG): container finished" podID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerID="432eab62005f7b54d460e4f1a5048aed82fe3a73c882e8266deadadc7e6e7695" exitCode=0 Jul 13 19:24:27.821078 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.820874 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:27.821078 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.820874 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57caca18-ea13-445b-a21a-016bb5a0f9c0","Type":"ContainerDied","Data":"432eab62005f7b54d460e4f1a5048aed82fe3a73c882e8266deadadc7e6e7695"} Jul 13 19:24:27.821078 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.820953 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57caca18-ea13-445b-a21a-016bb5a0f9c0","Type":"ContainerDied","Data":"7562ee0c27032504efd0b520be89a667140a35725d1fbc5523df9a6ead863bd5"} Jul 13 19:24:27.821078 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.820993 2556 scope.go:117] "RemoveContainer" containerID="9f8b087a6e49cd425e2c4f4fcb01b40577984245f478b11b74e676c1f10d6c26" Jul 13 19:24:27.829474 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.829453 2556 scope.go:117] "RemoveContainer" containerID="d80571a0b2bcba93e5bd22284dc53272da39a0c2c12ccbfdb756c1f45caf373b" Jul 13 19:24:27.833257 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.833233 2556 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57caca18-ea13-445b-a21a-016bb5a0f9c0-config-out\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:24:27.833257 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.833257 2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:24:27.833391 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.833269 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6qxt\" (UniqueName: \"kubernetes.io/projected/57caca18-ea13-445b-a21a-016bb5a0f9c0-kube-api-access-v6qxt\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:24:27.833391 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.833280 2556 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57caca18-ea13-445b-a21a-016bb5a0f9c0-tls-assets\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:24:27.833391 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.833289 2556 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-web-config\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:24:27.833391 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.833296 2556 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-config-volume\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:24:27.833391 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.833304 2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-main-tls\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:24:27.833391 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.833312 2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:24:27.833391 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.833320 2556 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-cluster-tls-config\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:24:27.833391 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.833329 2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57caca18-ea13-445b-a21a-016bb5a0f9c0-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:24:27.836945 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.836929 2556 scope.go:117] "RemoveContainer" containerID="fd35d134c7d35d868c3480c863f41e248a128172725c6422a0d67f6bcb990897" Jul 13 19:24:27.843526 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.843497 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 19:24:27.850635 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.848178 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 19:24:27.851213 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.851179 2556 scope.go:117] "RemoveContainer" containerID="432eab62005f7b54d460e4f1a5048aed82fe3a73c882e8266deadadc7e6e7695" Jul 13 19:24:27.859015 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.858973 2556 scope.go:117] "RemoveContainer" containerID="6304ed5b554dc01412e563cfd49a27fe2c586cb30410ab27297edaa75ef42665" Jul 13 19:24:27.867158 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.867138 2556 scope.go:117] "RemoveContainer" containerID="8183fdd1f287bcafcd419402e098a192cf614834b7c230bc83a84f166b1d240b" Jul 13 19:24:27.872215 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872190 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 19:24:27.872548 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872535 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="alertmanager" Jul 13 19:24:27.872594 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872551 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="alertmanager" Jul 13 19:24:27.872594 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872567 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="config-reloader" Jul 13 19:24:27.872594 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872573 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="config-reloader" Jul 13 19:24:27.872594 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872579 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="kube-rbac-proxy-web" Jul 13 19:24:27.872594 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872584 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="kube-rbac-proxy-web" Jul 13 19:24:27.872594 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872594 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="init-config-reloader" Jul 13 19:24:27.872766 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872600 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="init-config-reloader" Jul 13 19:24:27.872766 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872607 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="kube-rbac-proxy-metric" Jul 13 19:24:27.872766 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872612 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="kube-rbac-proxy-metric" Jul 13 19:24:27.872766 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872617 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="prom-label-proxy" Jul 13 19:24:27.872766 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872623 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="prom-label-proxy" Jul 13 19:24:27.872766 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872632 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="kube-rbac-proxy" Jul 13 19:24:27.872766 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872640 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="kube-rbac-proxy" Jul 13 19:24:27.872766 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872649 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36b77791-00e3-420e-8939-02e9fa1168ba" containerName="console" Jul 13 19:24:27.872766 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872654 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="36b77791-00e3-420e-8939-02e9fa1168ba" containerName="console" Jul 13 19:24:27.872766 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872696 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="config-reloader" Jul 13 19:24:27.872766 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872706 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="kube-rbac-proxy-metric" Jul 13 19:24:27.872766 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872716 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="prom-label-proxy" Jul 13 19:24:27.872766 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872722 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="36b77791-00e3-420e-8939-02e9fa1168ba" containerName="console" Jul 13 19:24:27.872766 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872727 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="kube-rbac-proxy" Jul 13 19:24:27.872766 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872732 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="alertmanager" Jul 13 19:24:27.872766 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.872737 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" containerName="kube-rbac-proxy-web" Jul 13 19:24:27.875435 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.875412 2556 scope.go:117] "RemoveContainer" containerID="d8a7ae2f6719662e4d133c8353620813468aa5607ac77c2c370ca42845b97643" Jul 13 19:24:27.876730 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.876713 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:27.878946 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.878883 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 13 19:24:27.878946 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.878900 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 13 19:24:27.878946 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.878910 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 13 19:24:27.879216 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.879035 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 13 19:24:27.879416 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.879398 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 13 19:24:27.879505 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.879442 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fms67\"" Jul 13 19:24:27.879505 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.879492 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 13 19:24:27.879611 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.879494 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 13 19:24:27.879611 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.879499 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 13 19:24:27.884135 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.884116 2556 scope.go:117] "RemoveContainer" containerID="9f8b087a6e49cd425e2c4f4fcb01b40577984245f478b11b74e676c1f10d6c26" Jul 13 19:24:27.884413 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:24:27.884393 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f8b087a6e49cd425e2c4f4fcb01b40577984245f478b11b74e676c1f10d6c26\": container with ID starting with 9f8b087a6e49cd425e2c4f4fcb01b40577984245f478b11b74e676c1f10d6c26 not found: ID does not exist" containerID="9f8b087a6e49cd425e2c4f4fcb01b40577984245f478b11b74e676c1f10d6c26" Jul 13 19:24:27.884489 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.884422 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f8b087a6e49cd425e2c4f4fcb01b40577984245f478b11b74e676c1f10d6c26"} err="failed to get container status \"9f8b087a6e49cd425e2c4f4fcb01b40577984245f478b11b74e676c1f10d6c26\": rpc error: code = NotFound desc = could not find container \"9f8b087a6e49cd425e2c4f4fcb01b40577984245f478b11b74e676c1f10d6c26\": container with ID starting with 9f8b087a6e49cd425e2c4f4fcb01b40577984245f478b11b74e676c1f10d6c26 not found: ID does not exist" Jul 13 19:24:27.884489 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.884445 2556 scope.go:117] "RemoveContainer" containerID="d80571a0b2bcba93e5bd22284dc53272da39a0c2c12ccbfdb756c1f45caf373b" Jul 13 19:24:27.884794 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:24:27.884729 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d80571a0b2bcba93e5bd22284dc53272da39a0c2c12ccbfdb756c1f45caf373b\": container with ID starting with d80571a0b2bcba93e5bd22284dc53272da39a0c2c12ccbfdb756c1f45caf373b not found: ID does not exist" containerID="d80571a0b2bcba93e5bd22284dc53272da39a0c2c12ccbfdb756c1f45caf373b" Jul 13 19:24:27.884878 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.884796 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d80571a0b2bcba93e5bd22284dc53272da39a0c2c12ccbfdb756c1f45caf373b"} err="failed to get container status \"d80571a0b2bcba93e5bd22284dc53272da39a0c2c12ccbfdb756c1f45caf373b\": rpc error: code = NotFound desc = could not find container \"d80571a0b2bcba93e5bd22284dc53272da39a0c2c12ccbfdb756c1f45caf373b\": container with ID starting with d80571a0b2bcba93e5bd22284dc53272da39a0c2c12ccbfdb756c1f45caf373b not found: ID does not exist" Jul 13 19:24:27.884878 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.884822 2556 scope.go:117] "RemoveContainer" containerID="fd35d134c7d35d868c3480c863f41e248a128172725c6422a0d67f6bcb990897" Jul 13 19:24:27.885137 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:24:27.885109 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd35d134c7d35d868c3480c863f41e248a128172725c6422a0d67f6bcb990897\": container with ID starting with fd35d134c7d35d868c3480c863f41e248a128172725c6422a0d67f6bcb990897 not found: ID does not exist" containerID="fd35d134c7d35d868c3480c863f41e248a128172725c6422a0d67f6bcb990897" Jul 13 19:24:27.885205 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.885147 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd35d134c7d35d868c3480c863f41e248a128172725c6422a0d67f6bcb990897"} err="failed to get container status \"fd35d134c7d35d868c3480c863f41e248a128172725c6422a0d67f6bcb990897\": rpc error: code = NotFound desc = could not find container \"fd35d134c7d35d868c3480c863f41e248a128172725c6422a0d67f6bcb990897\": container with ID starting with fd35d134c7d35d868c3480c863f41e248a128172725c6422a0d67f6bcb990897 not found: ID does not exist" Jul 13 19:24:27.885205 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.885172 2556 scope.go:117] "RemoveContainer" containerID="432eab62005f7b54d460e4f1a5048aed82fe3a73c882e8266deadadc7e6e7695" Jul 13 19:24:27.885526 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:24:27.885498 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"432eab62005f7b54d460e4f1a5048aed82fe3a73c882e8266deadadc7e6e7695\": container with ID starting with 432eab62005f7b54d460e4f1a5048aed82fe3a73c882e8266deadadc7e6e7695 not found: ID does not exist" containerID="432eab62005f7b54d460e4f1a5048aed82fe3a73c882e8266deadadc7e6e7695" Jul 13 19:24:27.885593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.885533 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"432eab62005f7b54d460e4f1a5048aed82fe3a73c882e8266deadadc7e6e7695"} err="failed to get container status \"432eab62005f7b54d460e4f1a5048aed82fe3a73c882e8266deadadc7e6e7695\": rpc error: code = NotFound desc = could not find container \"432eab62005f7b54d460e4f1a5048aed82fe3a73c882e8266deadadc7e6e7695\": container with ID starting with 432eab62005f7b54d460e4f1a5048aed82fe3a73c882e8266deadadc7e6e7695 not found: ID does not exist" Jul 13 19:24:27.885593 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.885552 2556 scope.go:117] "RemoveContainer" containerID="6304ed5b554dc01412e563cfd49a27fe2c586cb30410ab27297edaa75ef42665" Jul 13 19:24:27.885843 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:24:27.885820 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6304ed5b554dc01412e563cfd49a27fe2c586cb30410ab27297edaa75ef42665\": container with ID starting with 6304ed5b554dc01412e563cfd49a27fe2c586cb30410ab27297edaa75ef42665 not found: ID does not exist" containerID="6304ed5b554dc01412e563cfd49a27fe2c586cb30410ab27297edaa75ef42665" Jul 13 19:24:27.885906 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.885847 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6304ed5b554dc01412e563cfd49a27fe2c586cb30410ab27297edaa75ef42665"} err="failed to get container status \"6304ed5b554dc01412e563cfd49a27fe2c586cb30410ab27297edaa75ef42665\": rpc error: code = NotFound desc = could not find container \"6304ed5b554dc01412e563cfd49a27fe2c586cb30410ab27297edaa75ef42665\": container with ID starting with 6304ed5b554dc01412e563cfd49a27fe2c586cb30410ab27297edaa75ef42665 not found: ID does not exist" Jul 13 19:24:27.885906 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.885862 2556 scope.go:117] "RemoveContainer" containerID="8183fdd1f287bcafcd419402e098a192cf614834b7c230bc83a84f166b1d240b" Jul 13 19:24:27.886129 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:24:27.886114 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8183fdd1f287bcafcd419402e098a192cf614834b7c230bc83a84f166b1d240b\": container with ID starting with 8183fdd1f287bcafcd419402e098a192cf614834b7c230bc83a84f166b1d240b not found: ID does not exist" containerID="8183fdd1f287bcafcd419402e098a192cf614834b7c230bc83a84f166b1d240b" Jul 13 19:24:27.886172 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.886132 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8183fdd1f287bcafcd419402e098a192cf614834b7c230bc83a84f166b1d240b"} err="failed to get container status \"8183fdd1f287bcafcd419402e098a192cf614834b7c230bc83a84f166b1d240b\": rpc error: code = NotFound desc = could not find container \"8183fdd1f287bcafcd419402e098a192cf614834b7c230bc83a84f166b1d240b\": container with ID starting with 8183fdd1f287bcafcd419402e098a192cf614834b7c230bc83a84f166b1d240b not found: ID does not exist" Jul 13 19:24:27.886172 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.886146 2556 scope.go:117] "RemoveContainer" containerID="d8a7ae2f6719662e4d133c8353620813468aa5607ac77c2c370ca42845b97643" Jul 13 19:24:27.886400 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:24:27.886380 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8a7ae2f6719662e4d133c8353620813468aa5607ac77c2c370ca42845b97643\": container with ID starting with d8a7ae2f6719662e4d133c8353620813468aa5607ac77c2c370ca42845b97643 not found: ID does not exist" containerID="d8a7ae2f6719662e4d133c8353620813468aa5607ac77c2c370ca42845b97643" Jul 13 19:24:27.886448 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.886408 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8a7ae2f6719662e4d133c8353620813468aa5607ac77c2c370ca42845b97643"} err="failed to get container status \"d8a7ae2f6719662e4d133c8353620813468aa5607ac77c2c370ca42845b97643\": rpc error: code = NotFound desc = could not find container \"d8a7ae2f6719662e4d133c8353620813468aa5607ac77c2c370ca42845b97643\": container with ID starting with d8a7ae2f6719662e4d133c8353620813468aa5607ac77c2c370ca42845b97643 not found: ID does not exist" Jul 13 19:24:27.886716 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.886698 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 13 19:24:27.894846 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.894820 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 19:24:27.933792 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.933742 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43da95a6-a681-46f7-902e-6e206d0f299f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:27.933968 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.933802 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtb7v\" (UniqueName: \"kubernetes.io/projected/43da95a6-a681-46f7-902e-6e206d0f299f-kube-api-access-xtb7v\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:27.933968 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.933882 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:27.933968 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.933949 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-web-config\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:27.934066 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.933975 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:27.934066 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.933995 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43da95a6-a681-46f7-902e-6e206d0f299f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:27.934066 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.934013 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43da95a6-a681-46f7-902e-6e206d0f299f-config-out\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:27.934156 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.934056 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/43da95a6-a681-46f7-902e-6e206d0f299f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:27.934156 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.934085 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:27.934156 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.934110 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:27.934156 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.934141 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-config-volume\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:27.934318 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.934166 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:27.934318 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:27.934185 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43da95a6-a681-46f7-902e-6e206d0f299f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.035463 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.035429 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43da95a6-a681-46f7-902e-6e206d0f299f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.035625 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.035470 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtb7v\" (UniqueName: \"kubernetes.io/projected/43da95a6-a681-46f7-902e-6e206d0f299f-kube-api-access-xtb7v\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.035625 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.035522 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.035625 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.035559 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-web-config\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.035625 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.035590 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.035625 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.035619 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43da95a6-a681-46f7-902e-6e206d0f299f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.035886 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.035647 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43da95a6-a681-46f7-902e-6e206d0f299f-config-out\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.035886 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.035674 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/43da95a6-a681-46f7-902e-6e206d0f299f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.035886 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.035703 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.035886 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.035734 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.035886 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.035762 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-config-volume\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.035886 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.035830 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.035886 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.035857 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43da95a6-a681-46f7-902e-6e206d0f299f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.036613 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.036579 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43da95a6-a681-46f7-902e-6e206d0f299f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.036913 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.036884 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43da95a6-a681-46f7-902e-6e206d0f299f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.038660 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.038543 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43da95a6-a681-46f7-902e-6e206d0f299f-config-out\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.039243 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.038814 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-web-config\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.039243 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.038934 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/43da95a6-a681-46f7-902e-6e206d0f299f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.039243 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.038964 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.039243 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.038970 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43da95a6-a681-46f7-902e-6e206d0f299f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.039243 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.038994 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.039243 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.039112 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.039463 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.039333 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-config-volume\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.039596 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.039575 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.040725 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.040707 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/43da95a6-a681-46f7-902e-6e206d0f299f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.044414 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.044389 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtb7v\" (UniqueName: \"kubernetes.io/projected/43da95a6-a681-46f7-902e-6e206d0f299f-kube-api-access-xtb7v\") pod \"alertmanager-main-0\" (UID: \"43da95a6-a681-46f7-902e-6e206d0f299f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.188845 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.188765 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 19:24:28.336857 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.336727 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 19:24:28.340637 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:24:28.340606 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43da95a6_a681_46f7_902e_6e206d0f299f.slice/crio-84345e5b5c6344a0376e2478102a976b3fc07eeedd53c9f2089d9bd7c0d6590d WatchSource:0}: Error finding container 84345e5b5c6344a0376e2478102a976b3fc07eeedd53c9f2089d9bd7c0d6590d: Status 404 returned error can't find the container with id 84345e5b5c6344a0376e2478102a976b3fc07eeedd53c9f2089d9bd7c0d6590d Jul 13 19:24:28.826848 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.826811 2556 generic.go:356] "Generic (PLEG): container finished" podID="43da95a6-a681-46f7-902e-6e206d0f299f" containerID="dfc82aa630d1a8508b800de22e66ef567aba4e3f3a83e83dec2b26071f3f1acc" exitCode=0 Jul 13 19:24:28.827340 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.826905 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43da95a6-a681-46f7-902e-6e206d0f299f","Type":"ContainerDied","Data":"dfc82aa630d1a8508b800de22e66ef567aba4e3f3a83e83dec2b26071f3f1acc"} Jul 13 19:24:28.827340 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.827004 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43da95a6-a681-46f7-902e-6e206d0f299f","Type":"ContainerStarted","Data":"84345e5b5c6344a0376e2478102a976b3fc07eeedd53c9f2089d9bd7c0d6590d"} Jul 13 19:24:28.849063 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:28.849011 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57caca18-ea13-445b-a21a-016bb5a0f9c0" path="/var/lib/kubelet/pods/57caca18-ea13-445b-a21a-016bb5a0f9c0/volumes" Jul 13 19:24:29.840390 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:29.840350 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43da95a6-a681-46f7-902e-6e206d0f299f","Type":"ContainerStarted","Data":"3d6ef5c92ad03456063aa88441608fe24febcac5086c5821e641cf7df6d7461f"} Jul 13 19:24:29.840868 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:29.840443 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43da95a6-a681-46f7-902e-6e206d0f299f","Type":"ContainerStarted","Data":"d69a5e98ce09e4e99c1b5a6a95659d7640a6e7f78343e3c0847a42197d074d98"} Jul 13 19:24:29.840868 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:29.840475 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43da95a6-a681-46f7-902e-6e206d0f299f","Type":"ContainerStarted","Data":"c37391cb68d21fa263459ed117c7aafe0dbed30be81f4aa938483fff1fbd2537"} Jul 13 19:24:29.840868 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:29.840501 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43da95a6-a681-46f7-902e-6e206d0f299f","Type":"ContainerStarted","Data":"799c4d2b19104d658c0218873a5d0597630a548b768ccfc1735796ea9a9de801"} Jul 13 19:24:29.840868 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:29.840523 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43da95a6-a681-46f7-902e-6e206d0f299f","Type":"ContainerStarted","Data":"6963dfe6c4ce0dce67d8fe49ef856698a28d7013f79d1d90641aee7a59840109"} Jul 13 19:24:29.840868 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:29.840546 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43da95a6-a681-46f7-902e-6e206d0f299f","Type":"ContainerStarted","Data":"e9852bb9e5abb3aec27249b9c9bf065d39b4ad6bde3a11b26f8a583f45ce29c0"} Jul 13 19:24:29.867214 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:29.867140 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.852737561 podStartE2EDuration="2.867115922s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.378361ms" isStatefulPod=false podCreationTimestamp="2026-07-13 19:24:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:24:29.866256332 +0000 UTC m=+164.980164040" watchObservedRunningTime="2026-07-13 19:24:29.867115922 +0000 UTC m=+164.981023632" Jul 13 19:24:30.410078 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.410031 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-656664b677-6trm6"] Jul 13 19:24:30.413059 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.413031 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.417041 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.415333 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 13 19:24:30.417041 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.415414 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-q5p65\"" Jul 13 19:24:30.417041 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.415699 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 13 19:24:30.417041 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.415958 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 13 19:24:30.417041 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.416097 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 13 19:24:30.417041 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.416159 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 13 19:24:30.420029 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.420005 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 13 19:24:30.426360 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.426316 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-656664b677-6trm6"] Jul 13 19:24:30.459928 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.459890 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-secret-telemeter-client\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.459928 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.459932 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.460186 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.459960 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-federate-client-tls\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.460186 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.460019 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-serving-certs-ca-bundle\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.460186 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.460045 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm44s\" (UniqueName: \"kubernetes.io/projected/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-kube-api-access-qm44s\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.460186 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.460125 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-metrics-client-ca\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.460360 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.460206 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-telemeter-client-tls\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.460360 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.460249 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.560989 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.560942 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-metrics-client-ca\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.561218 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.561005 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-telemeter-client-tls\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.561218 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.561035 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.561218 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.561065 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-secret-telemeter-client\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.561218 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.561092 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.561218 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.561121 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-federate-client-tls\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.561218 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.561175 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-serving-certs-ca-bundle\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.561218 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.561210 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm44s\" (UniqueName: \"kubernetes.io/projected/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-kube-api-access-qm44s\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.562210 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.562176 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.562355 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.562244 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-metrics-client-ca\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.562686 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.562658 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-serving-certs-ca-bundle\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.563957 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.563927 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-federate-client-tls\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.564197 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.564180 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.564412 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.564377 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-telemeter-client-tls\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.564465 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.564382 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-secret-telemeter-client\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.570003 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.569977 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm44s\" (UniqueName: \"kubernetes.io/projected/75d3158d-8e01-4a6c-8ca9-66d0a2f1077a-kube-api-access-qm44s\") pod \"telemeter-client-656664b677-6trm6\" (UID: \"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a\") " pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.725447 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.725350 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-656664b677-6trm6" Jul 13 19:24:30.900291 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:30.900198 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-656664b677-6trm6"] Jul 13 19:24:30.902621 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:24:30.902584 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75d3158d_8e01_4a6c_8ca9_66d0a2f1077a.slice/crio-ab562db0e1435efbd7c42e9c16615ad5eb3a55dbb02c7925ff0bb59672fd8222 WatchSource:0}: Error finding container ab562db0e1435efbd7c42e9c16615ad5eb3a55dbb02c7925ff0bb59672fd8222: Status 404 returned error can't find the container with id ab562db0e1435efbd7c42e9c16615ad5eb3a55dbb02c7925ff0bb59672fd8222 Jul 13 19:24:31.855879 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:31.855824 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-656664b677-6trm6" event={"ID":"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a","Type":"ContainerStarted","Data":"ab562db0e1435efbd7c42e9c16615ad5eb3a55dbb02c7925ff0bb59672fd8222"} Jul 13 19:24:32.866580 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:32.866536 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-656664b677-6trm6" event={"ID":"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a","Type":"ContainerStarted","Data":"3f94e224dfa904495b03f1ad6d5b05a7d9a731cccd64bf7a3d5047cfb4c38e45"} Jul 13 19:24:32.867122 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:32.866626 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-656664b677-6trm6" event={"ID":"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a","Type":"ContainerStarted","Data":"ca25159192e2a86674563a35f6da6f27324fe424fe0ebb7cbc96eaa933168bdd"} Jul 13 19:24:32.867122 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:32.866646 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-656664b677-6trm6" event={"ID":"75d3158d-8e01-4a6c-8ca9-66d0a2f1077a","Type":"ContainerStarted","Data":"087c7e9634ee176427f4e9289ac0211c389adc09c78e756cda603fc33ab54ce5"} Jul 13 19:24:32.890534 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:32.890474 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-656664b677-6trm6" podStartSLOduration=1.331860571 podStartE2EDuration="2.890454571s" totalImagesPullingTime="1.558594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:24:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:24:32.889175672 +0000 UTC m=+168.003083391" watchObservedRunningTime="2026-07-13 19:24:32.890454571 +0000 UTC m=+168.004362279" Jul 13 19:24:33.697879 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.697840 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-776c7cd49-mhg65"] Jul 13 19:24:33.700946 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.700919 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.713356 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.713324 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-776c7cd49-mhg65"] Jul 13 19:24:33.791715 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.791678 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-trusted-ca-bundle\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.792035 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.792010 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-console-config\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.792206 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.792181 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8db76ce1-d8a0-4845-af9b-04758ca27906-console-oauth-config\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.792327 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.792236 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f5tq\" (UniqueName: \"kubernetes.io/projected/8db76ce1-d8a0-4845-af9b-04758ca27906-kube-api-access-6f5tq\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.792389 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.792354 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-oauth-serving-cert\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.792473 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.792442 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8db76ce1-d8a0-4845-af9b-04758ca27906-console-serving-cert\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.792564 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.792494 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-service-ca\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.893126 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.893083 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8db76ce1-d8a0-4845-af9b-04758ca27906-console-oauth-config\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.893583 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.893136 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f5tq\" (UniqueName: \"kubernetes.io/projected/8db76ce1-d8a0-4845-af9b-04758ca27906-kube-api-access-6f5tq\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.893583 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.893488 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-oauth-serving-cert\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.893700 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.893638 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8db76ce1-d8a0-4845-af9b-04758ca27906-console-serving-cert\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.893754 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.893719 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-service-ca\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.893824 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.893755 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-trusted-ca-bundle\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.893881 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.893849 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-console-config\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.894339 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.894308 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-oauth-serving-cert\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.894542 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.894478 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-service-ca\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.894663 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.894603 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-console-config\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.894720 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.894698 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-trusted-ca-bundle\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.895984 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.895958 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8db76ce1-d8a0-4845-af9b-04758ca27906-console-oauth-config\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.896262 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.896245 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8db76ce1-d8a0-4845-af9b-04758ca27906-console-serving-cert\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:33.901073 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:33.901048 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f5tq\" (UniqueName: \"kubernetes.io/projected/8db76ce1-d8a0-4845-af9b-04758ca27906-kube-api-access-6f5tq\") pod \"console-776c7cd49-mhg65\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:34.014215 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:34.014100 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:34.157159 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:34.157132 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-776c7cd49-mhg65"] Jul 13 19:24:34.159663 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:24:34.159631 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db76ce1_d8a0_4845_af9b_04758ca27906.slice/crio-5b91b532063ebb11305d0c01331f86584b6737dfb935191f30177e04949fa520 WatchSource:0}: Error finding container 5b91b532063ebb11305d0c01331f86584b6737dfb935191f30177e04949fa520: Status 404 returned error can't find the container with id 5b91b532063ebb11305d0c01331f86584b6737dfb935191f30177e04949fa520 Jul 13 19:24:34.877660 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:34.877621 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-776c7cd49-mhg65" event={"ID":"8db76ce1-d8a0-4845-af9b-04758ca27906","Type":"ContainerStarted","Data":"ddbfecf78c66519239845da84fbc0a19da9db00edff82aa769b333511d9d406e"} Jul 13 19:24:34.877893 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:34.877675 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-776c7cd49-mhg65" event={"ID":"8db76ce1-d8a0-4845-af9b-04758ca27906","Type":"ContainerStarted","Data":"5b91b532063ebb11305d0c01331f86584b6737dfb935191f30177e04949fa520"} Jul 13 19:24:34.896476 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:34.896422 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-776c7cd49-mhg65" podStartSLOduration=1.896403345 podStartE2EDuration="1.896403345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:24:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:24:34.895079115 +0000 UTC m=+170.008986826" watchObservedRunningTime="2026-07-13 19:24:34.896403345 +0000 UTC m=+170.010311058" Jul 13 19:24:44.014989 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:44.014885 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:44.015533 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:44.015002 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:44.020092 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:44.020062 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:44.926950 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:44.926917 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:24:44.991153 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:24:44.991116 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f87c7ffb-44kt8"] Jul 13 19:25:10.012719 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.012651 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f87c7ffb-44kt8" podUID="969f35ca-651b-4a65-a6fb-51a94419bb8f" containerName="console" containerID="cri-o://47a04ac1dbd71409c3bba905ca1df031cd36212eec93f20936aa294fb724d866" gracePeriod=15 Jul 13 19:25:10.255641 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.255615 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:25:10.430520 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.430468 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-trusted-ca-bundle\") pod \"969f35ca-651b-4a65-a6fb-51a94419bb8f\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " Jul 13 19:25:10.430520 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.430529 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-service-ca\") pod \"969f35ca-651b-4a65-a6fb-51a94419bb8f\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " Jul 13 19:25:10.430824 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.430565 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjfs4\" (UniqueName: \"kubernetes.io/projected/969f35ca-651b-4a65-a6fb-51a94419bb8f-kube-api-access-rjfs4\") pod \"969f35ca-651b-4a65-a6fb-51a94419bb8f\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " Jul 13 19:25:10.430824 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.430692 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/969f35ca-651b-4a65-a6fb-51a94419bb8f-console-oauth-config\") pod \"969f35ca-651b-4a65-a6fb-51a94419bb8f\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " Jul 13 19:25:10.430824 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.430754 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-oauth-serving-cert\") pod \"969f35ca-651b-4a65-a6fb-51a94419bb8f\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " Jul 13 19:25:10.431051 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.430858 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/969f35ca-651b-4a65-a6fb-51a94419bb8f-console-serving-cert\") pod \"969f35ca-651b-4a65-a6fb-51a94419bb8f\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " Jul 13 19:25:10.431051 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.430891 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-console-config\") pod \"969f35ca-651b-4a65-a6fb-51a94419bb8f\" (UID: \"969f35ca-651b-4a65-a6fb-51a94419bb8f\") " Jul 13 19:25:10.431145 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.431057 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "969f35ca-651b-4a65-a6fb-51a94419bb8f" (UID: "969f35ca-651b-4a65-a6fb-51a94419bb8f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:25:10.431145 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.431069 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-service-ca" (OuterVolumeSpecName: "service-ca") pod "969f35ca-651b-4a65-a6fb-51a94419bb8f" (UID: "969f35ca-651b-4a65-a6fb-51a94419bb8f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:25:10.431145 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.431114 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "969f35ca-651b-4a65-a6fb-51a94419bb8f" (UID: "969f35ca-651b-4a65-a6fb-51a94419bb8f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:25:10.431394 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.431367 2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-trusted-ca-bundle\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:25:10.431394 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.431382 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-console-config" (OuterVolumeSpecName: "console-config") pod "969f35ca-651b-4a65-a6fb-51a94419bb8f" (UID: "969f35ca-651b-4a65-a6fb-51a94419bb8f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:25:10.431583 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.431397 2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-service-ca\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:25:10.431583 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.431431 2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-oauth-serving-cert\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:25:10.432988 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.432955 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/969f35ca-651b-4a65-a6fb-51a94419bb8f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "969f35ca-651b-4a65-a6fb-51a94419bb8f" (UID: "969f35ca-651b-4a65-a6fb-51a94419bb8f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:25:10.433084 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.432978 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/969f35ca-651b-4a65-a6fb-51a94419bb8f-kube-api-access-rjfs4" (OuterVolumeSpecName: "kube-api-access-rjfs4") pod "969f35ca-651b-4a65-a6fb-51a94419bb8f" (UID: "969f35ca-651b-4a65-a6fb-51a94419bb8f"). InnerVolumeSpecName "kube-api-access-rjfs4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:25:10.433084 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.433038 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/969f35ca-651b-4a65-a6fb-51a94419bb8f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "969f35ca-651b-4a65-a6fb-51a94419bb8f" (UID: "969f35ca-651b-4a65-a6fb-51a94419bb8f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:25:10.532555 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.532516 2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/969f35ca-651b-4a65-a6fb-51a94419bb8f-console-serving-cert\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:25:10.532555 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.532546 2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/969f35ca-651b-4a65-a6fb-51a94419bb8f-console-config\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:25:10.532555 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.532556 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjfs4\" (UniqueName: \"kubernetes.io/projected/969f35ca-651b-4a65-a6fb-51a94419bb8f-kube-api-access-rjfs4\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:25:10.532555 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:10.532565 2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/969f35ca-651b-4a65-a6fb-51a94419bb8f-console-oauth-config\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:25:11.031125 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:11.031088 2556 generic.go:356] "Generic (PLEG): container finished" podID="969f35ca-651b-4a65-a6fb-51a94419bb8f" containerID="47a04ac1dbd71409c3bba905ca1df031cd36212eec93f20936aa294fb724d866" exitCode=0 Jul 13 19:25:11.031125 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:11.031135 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f87c7ffb-44kt8" event={"ID":"969f35ca-651b-4a65-a6fb-51a94419bb8f","Type":"ContainerDied","Data":"47a04ac1dbd71409c3bba905ca1df031cd36212eec93f20936aa294fb724d866"} Jul 13 19:25:11.031672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:11.031151 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f87c7ffb-44kt8" Jul 13 19:25:11.031672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:11.031177 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f87c7ffb-44kt8" event={"ID":"969f35ca-651b-4a65-a6fb-51a94419bb8f","Type":"ContainerDied","Data":"48a1671c822b2c9afdfa51729d23888b07e6898e881f1a36ce8b9310834d2a12"} Jul 13 19:25:11.031672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:11.031199 2556 scope.go:117] "RemoveContainer" containerID="47a04ac1dbd71409c3bba905ca1df031cd36212eec93f20936aa294fb724d866" Jul 13 19:25:11.039703 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:11.039681 2556 scope.go:117] "RemoveContainer" containerID="47a04ac1dbd71409c3bba905ca1df031cd36212eec93f20936aa294fb724d866" Jul 13 19:25:11.040024 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:25:11.040002 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47a04ac1dbd71409c3bba905ca1df031cd36212eec93f20936aa294fb724d866\": container with ID starting with 47a04ac1dbd71409c3bba905ca1df031cd36212eec93f20936aa294fb724d866 not found: ID does not exist" containerID="47a04ac1dbd71409c3bba905ca1df031cd36212eec93f20936aa294fb724d866" Jul 13 19:25:11.040108 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:11.040039 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47a04ac1dbd71409c3bba905ca1df031cd36212eec93f20936aa294fb724d866"} err="failed to get container status \"47a04ac1dbd71409c3bba905ca1df031cd36212eec93f20936aa294fb724d866\": rpc error: code = NotFound desc = could not find container \"47a04ac1dbd71409c3bba905ca1df031cd36212eec93f20936aa294fb724d866\": container with ID starting with 47a04ac1dbd71409c3bba905ca1df031cd36212eec93f20936aa294fb724d866 not found: ID does not exist" Jul 13 19:25:11.046519 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:11.046489 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f87c7ffb-44kt8"] Jul 13 19:25:11.050160 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:11.050124 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f87c7ffb-44kt8"] Jul 13 19:25:12.848245 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:25:12.848207 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="969f35ca-651b-4a65-a6fb-51a94419bb8f" path="/var/lib/kubelet/pods/969f35ca-651b-4a65-a6fb-51a94419bb8f/volumes" Jul 13 19:26:03.790166 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.790120 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-786b65d94-fqdbs"] Jul 13 19:26:03.790649 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.790551 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="969f35ca-651b-4a65-a6fb-51a94419bb8f" containerName="console" Jul 13 19:26:03.790649 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.790566 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="969f35ca-651b-4a65-a6fb-51a94419bb8f" containerName="console" Jul 13 19:26:03.790649 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.790625 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="969f35ca-651b-4a65-a6fb-51a94419bb8f" containerName="console" Jul 13 19:26:03.793612 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.793589 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.797143 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.797113 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-trusted-ca-bundle\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.797300 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.797156 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/193869b9-d860-4865-b09f-09abd9d91e15-console-serving-cert\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.797300 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.797216 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5j6w\" (UniqueName: \"kubernetes.io/projected/193869b9-d860-4865-b09f-09abd9d91e15-kube-api-access-k5j6w\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.797300 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.797244 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-console-config\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.797300 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.797285 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-service-ca\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.797655 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.797316 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/193869b9-d860-4865-b09f-09abd9d91e15-console-oauth-config\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.797655 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.797351 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-oauth-serving-cert\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.812875 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.812841 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-786b65d94-fqdbs"] Jul 13 19:26:03.898217 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.898172 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/193869b9-d860-4865-b09f-09abd9d91e15-console-oauth-config\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.898217 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.898218 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-oauth-serving-cert\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.898471 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.898309 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-trusted-ca-bundle\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.898471 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.898341 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/193869b9-d860-4865-b09f-09abd9d91e15-console-serving-cert\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.898471 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.898439 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5j6w\" (UniqueName: \"kubernetes.io/projected/193869b9-d860-4865-b09f-09abd9d91e15-kube-api-access-k5j6w\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.898631 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.898497 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-console-config\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.898631 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.898541 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-service-ca\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.899279 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.899255 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-console-config\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.899279 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.899260 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-oauth-serving-cert\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.899432 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.899309 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-service-ca\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.899432 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.899390 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-trusted-ca-bundle\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.901162 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.901135 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/193869b9-d860-4865-b09f-09abd9d91e15-console-oauth-config\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.901292 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.901144 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/193869b9-d860-4865-b09f-09abd9d91e15-console-serving-cert\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:03.906056 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:03.906035 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5j6w\" (UniqueName: \"kubernetes.io/projected/193869b9-d860-4865-b09f-09abd9d91e15-kube-api-access-k5j6w\") pod \"console-786b65d94-fqdbs\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:04.104258 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:04.104148 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:04.244380 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:04.244345 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-786b65d94-fqdbs"] Jul 13 19:26:04.248737 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:26:04.248704 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod193869b9_d860_4865_b09f_09abd9d91e15.slice/crio-b0dce080ca7bfa5ec01d15646d77fc6a4248e04bab7533520da1000b2848d329 WatchSource:0}: Error finding container b0dce080ca7bfa5ec01d15646d77fc6a4248e04bab7533520da1000b2848d329: Status 404 returned error can't find the container with id b0dce080ca7bfa5ec01d15646d77fc6a4248e04bab7533520da1000b2848d329 Jul 13 19:26:04.258056 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:04.258029 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-786b65d94-fqdbs" event={"ID":"193869b9-d860-4865-b09f-09abd9d91e15","Type":"ContainerStarted","Data":"b0dce080ca7bfa5ec01d15646d77fc6a4248e04bab7533520da1000b2848d329"} Jul 13 19:26:05.269549 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:05.269500 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-786b65d94-fqdbs" event={"ID":"193869b9-d860-4865-b09f-09abd9d91e15","Type":"ContainerStarted","Data":"a9dfdf6179960f1d2f71cdad1de7e9157257813b064e3250bced3595d63fe529"} Jul 13 19:26:05.287288 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:05.287214 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-786b65d94-fqdbs" podStartSLOduration=2.28719309 podStartE2EDuration="2.28719309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:26:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:26:05.286960462 +0000 UTC m=+260.400868174" watchObservedRunningTime="2026-07-13 19:26:05.28719309 +0000 UTC m=+260.401100800" Jul 13 19:26:14.104574 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:14.104458 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:14.104574 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:14.104546 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:14.109531 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:14.109501 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:14.314155 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:14.314123 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:26:14.358139 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:14.358051 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-776c7cd49-mhg65"] Jul 13 19:26:39.381420 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.381365 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-776c7cd49-mhg65" podUID="8db76ce1-d8a0-4845-af9b-04758ca27906" containerName="console" containerID="cri-o://ddbfecf78c66519239845da84fbc0a19da9db00edff82aa769b333511d9d406e" gracePeriod=15 Jul 13 19:26:39.629557 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.629528 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:26:39.730278 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.730183 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6f5tq\" (UniqueName: \"kubernetes.io/projected/8db76ce1-d8a0-4845-af9b-04758ca27906-kube-api-access-6f5tq\") pod \"8db76ce1-d8a0-4845-af9b-04758ca27906\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " Jul 13 19:26:39.730278 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.730240 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8db76ce1-d8a0-4845-af9b-04758ca27906-console-serving-cert\") pod \"8db76ce1-d8a0-4845-af9b-04758ca27906\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " Jul 13 19:26:39.730278 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.730270 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-trusted-ca-bundle\") pod \"8db76ce1-d8a0-4845-af9b-04758ca27906\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " Jul 13 19:26:39.730550 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.730287 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8db76ce1-d8a0-4845-af9b-04758ca27906-console-oauth-config\") pod \"8db76ce1-d8a0-4845-af9b-04758ca27906\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " Jul 13 19:26:39.730550 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.730307 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-service-ca\") pod \"8db76ce1-d8a0-4845-af9b-04758ca27906\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " Jul 13 19:26:39.730550 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.730333 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-console-config\") pod \"8db76ce1-d8a0-4845-af9b-04758ca27906\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " Jul 13 19:26:39.730550 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.730360 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-oauth-serving-cert\") pod \"8db76ce1-d8a0-4845-af9b-04758ca27906\" (UID: \"8db76ce1-d8a0-4845-af9b-04758ca27906\") " Jul 13 19:26:39.730822 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.730760 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8db76ce1-d8a0-4845-af9b-04758ca27906" (UID: "8db76ce1-d8a0-4845-af9b-04758ca27906"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:26:39.730960 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.730825 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-service-ca" (OuterVolumeSpecName: "service-ca") pod "8db76ce1-d8a0-4845-af9b-04758ca27906" (UID: "8db76ce1-d8a0-4845-af9b-04758ca27906"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:26:39.730960 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.730842 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-console-config" (OuterVolumeSpecName: "console-config") pod "8db76ce1-d8a0-4845-af9b-04758ca27906" (UID: "8db76ce1-d8a0-4845-af9b-04758ca27906"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:26:39.731139 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.731001 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8db76ce1-d8a0-4845-af9b-04758ca27906" (UID: "8db76ce1-d8a0-4845-af9b-04758ca27906"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:26:39.732646 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.732615 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8db76ce1-d8a0-4845-af9b-04758ca27906-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8db76ce1-d8a0-4845-af9b-04758ca27906" (UID: "8db76ce1-d8a0-4845-af9b-04758ca27906"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:26:39.732727 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.732627 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8db76ce1-d8a0-4845-af9b-04758ca27906-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8db76ce1-d8a0-4845-af9b-04758ca27906" (UID: "8db76ce1-d8a0-4845-af9b-04758ca27906"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:26:39.732727 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.732627 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8db76ce1-d8a0-4845-af9b-04758ca27906-kube-api-access-6f5tq" (OuterVolumeSpecName: "kube-api-access-6f5tq") pod "8db76ce1-d8a0-4845-af9b-04758ca27906" (UID: "8db76ce1-d8a0-4845-af9b-04758ca27906"). InnerVolumeSpecName "kube-api-access-6f5tq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:26:39.831740 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.831699 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6f5tq\" (UniqueName: \"kubernetes.io/projected/8db76ce1-d8a0-4845-af9b-04758ca27906-kube-api-access-6f5tq\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:26:39.831740 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.831734 2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8db76ce1-d8a0-4845-af9b-04758ca27906-console-serving-cert\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:26:39.831740 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.831745 2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-trusted-ca-bundle\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:26:39.831740 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.831754 2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8db76ce1-d8a0-4845-af9b-04758ca27906-console-oauth-config\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:26:39.832048 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.831763 2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-service-ca\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:26:39.832048 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.831801 2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-console-config\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:26:39.832048 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:39.831810 2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8db76ce1-d8a0-4845-af9b-04758ca27906-oauth-serving-cert\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:26:40.430272 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:40.430225 2556 generic.go:356] "Generic (PLEG): container finished" podID="8db76ce1-d8a0-4845-af9b-04758ca27906" containerID="ddbfecf78c66519239845da84fbc0a19da9db00edff82aa769b333511d9d406e" exitCode=0 Jul 13 19:26:40.430272 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:40.430274 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-776c7cd49-mhg65" Jul 13 19:26:40.430800 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:40.430317 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-776c7cd49-mhg65" event={"ID":"8db76ce1-d8a0-4845-af9b-04758ca27906","Type":"ContainerDied","Data":"ddbfecf78c66519239845da84fbc0a19da9db00edff82aa769b333511d9d406e"} Jul 13 19:26:40.430800 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:40.430375 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-776c7cd49-mhg65" event={"ID":"8db76ce1-d8a0-4845-af9b-04758ca27906","Type":"ContainerDied","Data":"5b91b532063ebb11305d0c01331f86584b6737dfb935191f30177e04949fa520"} Jul 13 19:26:40.430800 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:40.430408 2556 scope.go:117] "RemoveContainer" containerID="ddbfecf78c66519239845da84fbc0a19da9db00edff82aa769b333511d9d406e" Jul 13 19:26:40.440183 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:40.440162 2556 scope.go:117] "RemoveContainer" containerID="ddbfecf78c66519239845da84fbc0a19da9db00edff82aa769b333511d9d406e" Jul 13 19:26:40.440586 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:26:40.440560 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddbfecf78c66519239845da84fbc0a19da9db00edff82aa769b333511d9d406e\": container with ID starting with ddbfecf78c66519239845da84fbc0a19da9db00edff82aa769b333511d9d406e not found: ID does not exist" containerID="ddbfecf78c66519239845da84fbc0a19da9db00edff82aa769b333511d9d406e" Jul 13 19:26:40.440702 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:40.440592 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddbfecf78c66519239845da84fbc0a19da9db00edff82aa769b333511d9d406e"} err="failed to get container status \"ddbfecf78c66519239845da84fbc0a19da9db00edff82aa769b333511d9d406e\": rpc error: code = NotFound desc = could not find container \"ddbfecf78c66519239845da84fbc0a19da9db00edff82aa769b333511d9d406e\": container with ID starting with ddbfecf78c66519239845da84fbc0a19da9db00edff82aa769b333511d9d406e not found: ID does not exist" Jul 13 19:26:40.455863 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:40.455827 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-776c7cd49-mhg65"] Jul 13 19:26:40.459642 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:40.459612 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-776c7cd49-mhg65"] Jul 13 19:26:40.848000 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:40.847958 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8db76ce1-d8a0-4845-af9b-04758ca27906" path="/var/lib/kubelet/pods/8db76ce1-d8a0-4845-af9b-04758ca27906/volumes" Jul 13 19:26:45.809529 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:45.809501 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 19:26:45.810408 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:45.810372 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 19:26:45.817108 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:45.817084 2556 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 19:26:54.633101 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.633059 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg"] Jul 13 19:26:54.634420 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.633447 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8db76ce1-d8a0-4845-af9b-04758ca27906" containerName="console" Jul 13 19:26:54.634420 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.633460 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8db76ce1-d8a0-4845-af9b-04758ca27906" containerName="console" Jul 13 19:26:54.634420 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.633543 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8db76ce1-d8a0-4845-af9b-04758ca27906" containerName="console" Jul 13 19:26:54.635745 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.635726 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg" Jul 13 19:26:54.638035 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.638015 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 19:26:54.638535 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.638507 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 19:26:54.638646 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.638593 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rkn6h\"" Jul 13 19:26:54.645885 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.645857 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg"] Jul 13 19:26:54.655768 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.655736 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjjj7\" (UniqueName: \"kubernetes.io/projected/f4f4a235-e9dd-493c-8130-058ed78f99a8-kube-api-access-gjjj7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg\" (UID: \"f4f4a235-e9dd-493c-8130-058ed78f99a8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg" Jul 13 19:26:54.655951 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.655804 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4f4a235-e9dd-493c-8130-058ed78f99a8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg\" (UID: \"f4f4a235-e9dd-493c-8130-058ed78f99a8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg" Jul 13 19:26:54.655951 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.655844 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4f4a235-e9dd-493c-8130-058ed78f99a8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg\" (UID: \"f4f4a235-e9dd-493c-8130-058ed78f99a8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg" Jul 13 19:26:54.756990 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.756953 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4f4a235-e9dd-493c-8130-058ed78f99a8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg\" (UID: \"f4f4a235-e9dd-493c-8130-058ed78f99a8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg" Jul 13 19:26:54.757166 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.757016 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4f4a235-e9dd-493c-8130-058ed78f99a8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg\" (UID: \"f4f4a235-e9dd-493c-8130-058ed78f99a8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg" Jul 13 19:26:54.757166 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.757062 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjjj7\" (UniqueName: \"kubernetes.io/projected/f4f4a235-e9dd-493c-8130-058ed78f99a8-kube-api-access-gjjj7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg\" (UID: \"f4f4a235-e9dd-493c-8130-058ed78f99a8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg" Jul 13 19:26:54.757433 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.757411 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4f4a235-e9dd-493c-8130-058ed78f99a8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg\" (UID: \"f4f4a235-e9dd-493c-8130-058ed78f99a8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg" Jul 13 19:26:54.757433 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.757423 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4f4a235-e9dd-493c-8130-058ed78f99a8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg\" (UID: \"f4f4a235-e9dd-493c-8130-058ed78f99a8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg" Jul 13 19:26:54.765315 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.765279 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjjj7\" (UniqueName: \"kubernetes.io/projected/f4f4a235-e9dd-493c-8130-058ed78f99a8-kube-api-access-gjjj7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg\" (UID: \"f4f4a235-e9dd-493c-8130-058ed78f99a8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg" Jul 13 19:26:54.945974 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:54.945886 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg" Jul 13 19:26:55.082420 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:55.082387 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg"] Jul 13 19:26:55.085271 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:26:55.085242 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f4a235_e9dd_493c_8130_058ed78f99a8.slice/crio-25d7760abf4e6bd941d17c5896ed0bde59582a115b91bf1b524a89ee2baca1d3 WatchSource:0}: Error finding container 25d7760abf4e6bd941d17c5896ed0bde59582a115b91bf1b524a89ee2baca1d3: Status 404 returned error can't find the container with id 25d7760abf4e6bd941d17c5896ed0bde59582a115b91bf1b524a89ee2baca1d3 Jul 13 19:26:55.087787 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:55.087753 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 19:26:55.497424 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:26:55.497384 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg" event={"ID":"f4f4a235-e9dd-493c-8130-058ed78f99a8","Type":"ContainerStarted","Data":"25d7760abf4e6bd941d17c5896ed0bde59582a115b91bf1b524a89ee2baca1d3"} Jul 13 19:27:04.541616 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:04.541578 2556 generic.go:356] "Generic (PLEG): container finished" podID="f4f4a235-e9dd-493c-8130-058ed78f99a8" containerID="26270a8db90cb6a37820a40ddc755455ff4e9a834f8d2751b77308487a6778f6" exitCode=0 Jul 13 19:27:04.542072 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:04.541671 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg" event={"ID":"f4f4a235-e9dd-493c-8130-058ed78f99a8","Type":"ContainerDied","Data":"26270a8db90cb6a37820a40ddc755455ff4e9a834f8d2751b77308487a6778f6"} Jul 13 19:27:07.557373 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:07.557338 2556 generic.go:356] "Generic (PLEG): container finished" podID="f4f4a235-e9dd-493c-8130-058ed78f99a8" containerID="2c5540fbb0832541be5fa4c4c27941d4310b3dfca9aa6a70fb424297f867884b" exitCode=0 Jul 13 19:27:07.557846 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:07.557433 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg" event={"ID":"f4f4a235-e9dd-493c-8130-058ed78f99a8","Type":"ContainerDied","Data":"2c5540fbb0832541be5fa4c4c27941d4310b3dfca9aa6a70fb424297f867884b"} Jul 13 19:27:18.611874 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:18.611841 2556 generic.go:356] "Generic (PLEG): container finished" podID="f4f4a235-e9dd-493c-8130-058ed78f99a8" containerID="ab7639b0ad65dbb0f8ce768e3e62176fcd8f55daf6481b5fd7b2edb2aaad0bc7" exitCode=0 Jul 13 19:27:18.612649 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:18.611913 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg" event={"ID":"f4f4a235-e9dd-493c-8130-058ed78f99a8","Type":"ContainerDied","Data":"ab7639b0ad65dbb0f8ce768e3e62176fcd8f55daf6481b5fd7b2edb2aaad0bc7"} Jul 13 19:27:19.753991 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:19.753965 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg" Jul 13 19:27:19.895502 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:19.895406 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4f4a235-e9dd-493c-8130-058ed78f99a8-util\") pod \"f4f4a235-e9dd-493c-8130-058ed78f99a8\" (UID: \"f4f4a235-e9dd-493c-8130-058ed78f99a8\") " Jul 13 19:27:19.895502 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:19.895472 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjjj7\" (UniqueName: \"kubernetes.io/projected/f4f4a235-e9dd-493c-8130-058ed78f99a8-kube-api-access-gjjj7\") pod \"f4f4a235-e9dd-493c-8130-058ed78f99a8\" (UID: \"f4f4a235-e9dd-493c-8130-058ed78f99a8\") " Jul 13 19:27:19.895699 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:19.895530 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4f4a235-e9dd-493c-8130-058ed78f99a8-bundle\") pod \"f4f4a235-e9dd-493c-8130-058ed78f99a8\" (UID: \"f4f4a235-e9dd-493c-8130-058ed78f99a8\") " Jul 13 19:27:19.896242 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:19.896213 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4f4a235-e9dd-493c-8130-058ed78f99a8-bundle" (OuterVolumeSpecName: "bundle") pod "f4f4a235-e9dd-493c-8130-058ed78f99a8" (UID: "f4f4a235-e9dd-493c-8130-058ed78f99a8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:27:19.897841 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:19.897810 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4f4a235-e9dd-493c-8130-058ed78f99a8-kube-api-access-gjjj7" (OuterVolumeSpecName: "kube-api-access-gjjj7") pod "f4f4a235-e9dd-493c-8130-058ed78f99a8" (UID: "f4f4a235-e9dd-493c-8130-058ed78f99a8"). InnerVolumeSpecName "kube-api-access-gjjj7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:27:19.902346 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:19.902313 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4f4a235-e9dd-493c-8130-058ed78f99a8-util" (OuterVolumeSpecName: "util") pod "f4f4a235-e9dd-493c-8130-058ed78f99a8" (UID: "f4f4a235-e9dd-493c-8130-058ed78f99a8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:27:19.997175 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:19.997134 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjjj7\" (UniqueName: \"kubernetes.io/projected/f4f4a235-e9dd-493c-8130-058ed78f99a8-kube-api-access-gjjj7\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:27:19.997175 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:19.997170 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4f4a235-e9dd-493c-8130-058ed78f99a8-bundle\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:27:19.997175 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:19.997180 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4f4a235-e9dd-493c-8130-058ed78f99a8-util\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:27:20.623624 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:20.623586 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg" event={"ID":"f4f4a235-e9dd-493c-8130-058ed78f99a8","Type":"ContainerDied","Data":"25d7760abf4e6bd941d17c5896ed0bde59582a115b91bf1b524a89ee2baca1d3"} Jul 13 19:27:20.623788 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:20.623642 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25d7760abf4e6bd941d17c5896ed0bde59582a115b91bf1b524a89ee2baca1d3" Jul 13 19:27:20.623788 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:20.623600 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540p6wbg" Jul 13 19:27:27.957936 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:27.957902 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wsltv"] Jul 13 19:27:27.958347 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:27.958264 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4f4a235-e9dd-493c-8130-058ed78f99a8" containerName="util" Jul 13 19:27:27.958347 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:27.958275 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4f4a235-e9dd-493c-8130-058ed78f99a8" containerName="util" Jul 13 19:27:27.958347 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:27.958287 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4f4a235-e9dd-493c-8130-058ed78f99a8" containerName="pull" Jul 13 19:27:27.958347 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:27.958293 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4f4a235-e9dd-493c-8130-058ed78f99a8" containerName="pull" Jul 13 19:27:27.958347 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:27.958305 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4f4a235-e9dd-493c-8130-058ed78f99a8" containerName="extract" Jul 13 19:27:27.958347 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:27.958311 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4f4a235-e9dd-493c-8130-058ed78f99a8" containerName="extract" Jul 13 19:27:27.958532 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:27.958367 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4f4a235-e9dd-493c-8130-058ed78f99a8" containerName="extract" Jul 13 19:27:27.960622 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:27.960605 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wsltv" Jul 13 19:27:27.964437 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:27.964410 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 13 19:27:27.964585 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:27.964410 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 13 19:27:27.964585 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:27.964411 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-5qrdh\"" Jul 13 19:27:27.975546 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:27.975517 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wsltv"] Jul 13 19:27:28.073025 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:28.072985 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11fb9e4b-5b8f-4556-82cb-751815d3c398-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wsltv\" (UID: \"11fb9e4b-5b8f-4556-82cb-751815d3c398\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wsltv" Jul 13 19:27:28.073025 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:28.073038 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7ckx\" (UniqueName: \"kubernetes.io/projected/11fb9e4b-5b8f-4556-82cb-751815d3c398-kube-api-access-b7ckx\") pod \"cert-manager-operator-controller-manager-55c444bf86-wsltv\" (UID: \"11fb9e4b-5b8f-4556-82cb-751815d3c398\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wsltv" Jul 13 19:27:28.173912 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:28.173867 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11fb9e4b-5b8f-4556-82cb-751815d3c398-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wsltv\" (UID: \"11fb9e4b-5b8f-4556-82cb-751815d3c398\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wsltv" Jul 13 19:27:28.174096 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:28.173923 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7ckx\" (UniqueName: \"kubernetes.io/projected/11fb9e4b-5b8f-4556-82cb-751815d3c398-kube-api-access-b7ckx\") pod \"cert-manager-operator-controller-manager-55c444bf86-wsltv\" (UID: \"11fb9e4b-5b8f-4556-82cb-751815d3c398\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wsltv" Jul 13 19:27:28.174383 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:28.174357 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11fb9e4b-5b8f-4556-82cb-751815d3c398-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wsltv\" (UID: \"11fb9e4b-5b8f-4556-82cb-751815d3c398\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wsltv" Jul 13 19:27:28.182260 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:28.182230 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7ckx\" (UniqueName: \"kubernetes.io/projected/11fb9e4b-5b8f-4556-82cb-751815d3c398-kube-api-access-b7ckx\") pod \"cert-manager-operator-controller-manager-55c444bf86-wsltv\" (UID: \"11fb9e4b-5b8f-4556-82cb-751815d3c398\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wsltv" Jul 13 19:27:28.270941 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:28.270847 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wsltv" Jul 13 19:27:28.412174 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:28.412148 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wsltv"] Jul 13 19:27:28.415316 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:27:28.415277 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11fb9e4b_5b8f_4556_82cb_751815d3c398.slice/crio-405ed2282c146a7d01a03105901fdeeefdd992478d8916f5e6f68bb24ff76be0 WatchSource:0}: Error finding container 405ed2282c146a7d01a03105901fdeeefdd992478d8916f5e6f68bb24ff76be0: Status 404 returned error can't find the container with id 405ed2282c146a7d01a03105901fdeeefdd992478d8916f5e6f68bb24ff76be0 Jul 13 19:27:28.661825 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:28.661770 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wsltv" event={"ID":"11fb9e4b-5b8f-4556-82cb-751815d3c398","Type":"ContainerStarted","Data":"405ed2282c146a7d01a03105901fdeeefdd992478d8916f5e6f68bb24ff76be0"} Jul 13 19:27:30.674616 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:30.674577 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wsltv" event={"ID":"11fb9e4b-5b8f-4556-82cb-751815d3c398","Type":"ContainerStarted","Data":"c8879138600c9db5bcedb4a6748377ee5a4d78203f65f84238b39568a2123228"} Jul 13 19:27:30.705550 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:30.705480 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wsltv" podStartSLOduration=1.951927908 podStartE2EDuration="3.705457552s" totalImagesPullingTime="1.753529644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:27:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:27:30.704312876 +0000 UTC m=+345.818220622" watchObservedRunningTime="2026-07-13 19:27:30.705457552 +0000 UTC m=+345.819365266" Jul 13 19:27:32.169319 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:32.169284 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z"] Jul 13 19:27:32.172797 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:32.172754 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" Jul 13 19:27:32.177644 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:32.177619 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rkn6h\"" Jul 13 19:27:32.177762 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:32.177722 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 19:27:32.178056 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:32.178041 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 19:27:32.193410 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:32.193367 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z"] Jul 13 19:27:32.317086 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:32.317046 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69ml5\" (UniqueName: \"kubernetes.io/projected/1156b07e-2f94-4625-81c9-e9c0647d17e0-kube-api-access-69ml5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z\" (UID: \"1156b07e-2f94-4625-81c9-e9c0647d17e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" Jul 13 19:27:32.317086 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:32.317093 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1156b07e-2f94-4625-81c9-e9c0647d17e0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z\" (UID: \"1156b07e-2f94-4625-81c9-e9c0647d17e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" Jul 13 19:27:32.317313 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:32.317210 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1156b07e-2f94-4625-81c9-e9c0647d17e0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z\" (UID: \"1156b07e-2f94-4625-81c9-e9c0647d17e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" Jul 13 19:27:32.417929 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:32.417889 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1156b07e-2f94-4625-81c9-e9c0647d17e0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z\" (UID: \"1156b07e-2f94-4625-81c9-e9c0647d17e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" Jul 13 19:27:32.418130 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:32.417976 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69ml5\" (UniqueName: \"kubernetes.io/projected/1156b07e-2f94-4625-81c9-e9c0647d17e0-kube-api-access-69ml5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z\" (UID: \"1156b07e-2f94-4625-81c9-e9c0647d17e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" Jul 13 19:27:32.418130 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:32.418001 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1156b07e-2f94-4625-81c9-e9c0647d17e0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z\" (UID: \"1156b07e-2f94-4625-81c9-e9c0647d17e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" Jul 13 19:27:32.418320 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:32.418299 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1156b07e-2f94-4625-81c9-e9c0647d17e0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z\" (UID: \"1156b07e-2f94-4625-81c9-e9c0647d17e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" Jul 13 19:27:32.418392 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:32.418371 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1156b07e-2f94-4625-81c9-e9c0647d17e0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z\" (UID: \"1156b07e-2f94-4625-81c9-e9c0647d17e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" Jul 13 19:27:32.437425 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:32.437366 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69ml5\" (UniqueName: \"kubernetes.io/projected/1156b07e-2f94-4625-81c9-e9c0647d17e0-kube-api-access-69ml5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z\" (UID: \"1156b07e-2f94-4625-81c9-e9c0647d17e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" Jul 13 19:27:32.486979 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:32.486931 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" Jul 13 19:27:32.628024 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:32.627995 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z"] Jul 13 19:27:32.630622 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:27:32.630574 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1156b07e_2f94_4625_81c9_e9c0647d17e0.slice/crio-36cbf2ab7e4ead1dd02c1436dc0fa16665d5b42515cc7eaad802196ae9eb80f0 WatchSource:0}: Error finding container 36cbf2ab7e4ead1dd02c1436dc0fa16665d5b42515cc7eaad802196ae9eb80f0: Status 404 returned error can't find the container with id 36cbf2ab7e4ead1dd02c1436dc0fa16665d5b42515cc7eaad802196ae9eb80f0 Jul 13 19:27:32.685230 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:32.685193 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" event={"ID":"1156b07e-2f94-4625-81c9-e9c0647d17e0","Type":"ContainerStarted","Data":"36cbf2ab7e4ead1dd02c1436dc0fa16665d5b42515cc7eaad802196ae9eb80f0"} Jul 13 19:27:33.319883 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:33.319840 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9vgdf"] Jul 13 19:27:33.322818 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:33.322789 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9vgdf" Jul 13 19:27:33.325067 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:33.325048 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 13 19:27:33.325497 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:33.325479 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 13 19:27:33.325561 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:33.325527 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-44cnx\"" Jul 13 19:27:33.337692 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:33.337660 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9vgdf"] Jul 13 19:27:33.426611 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:33.426570 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqtmq\" (UniqueName: \"kubernetes.io/projected/27d03ef3-472d-41a1-9d3a-1ccd7f9b7eca-kube-api-access-xqtmq\") pod \"cert-manager-webhook-6486ffd997-9vgdf\" (UID: \"27d03ef3-472d-41a1-9d3a-1ccd7f9b7eca\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9vgdf" Jul 13 19:27:33.426817 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:33.426698 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27d03ef3-472d-41a1-9d3a-1ccd7f9b7eca-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9vgdf\" (UID: \"27d03ef3-472d-41a1-9d3a-1ccd7f9b7eca\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9vgdf" Jul 13 19:27:33.528039 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:33.527994 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27d03ef3-472d-41a1-9d3a-1ccd7f9b7eca-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9vgdf\" (UID: \"27d03ef3-472d-41a1-9d3a-1ccd7f9b7eca\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9vgdf" Jul 13 19:27:33.528198 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:33.528057 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqtmq\" (UniqueName: \"kubernetes.io/projected/27d03ef3-472d-41a1-9d3a-1ccd7f9b7eca-kube-api-access-xqtmq\") pod \"cert-manager-webhook-6486ffd997-9vgdf\" (UID: \"27d03ef3-472d-41a1-9d3a-1ccd7f9b7eca\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9vgdf" Jul 13 19:27:33.537305 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:33.537271 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27d03ef3-472d-41a1-9d3a-1ccd7f9b7eca-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9vgdf\" (UID: \"27d03ef3-472d-41a1-9d3a-1ccd7f9b7eca\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9vgdf" Jul 13 19:27:33.537456 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:33.537405 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqtmq\" (UniqueName: \"kubernetes.io/projected/27d03ef3-472d-41a1-9d3a-1ccd7f9b7eca-kube-api-access-xqtmq\") pod \"cert-manager-webhook-6486ffd997-9vgdf\" (UID: \"27d03ef3-472d-41a1-9d3a-1ccd7f9b7eca\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9vgdf" Jul 13 19:27:33.650011 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:33.649913 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9vgdf" Jul 13 19:27:33.692210 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:33.692170 2556 generic.go:356] "Generic (PLEG): container finished" podID="1156b07e-2f94-4625-81c9-e9c0647d17e0" containerID="88456ad52e9093c0171420758cf681dd493dd8fcf65dc7cf32f512caf7405426" exitCode=0 Jul 13 19:27:33.692393 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:33.692255 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" event={"ID":"1156b07e-2f94-4625-81c9-e9c0647d17e0","Type":"ContainerDied","Data":"88456ad52e9093c0171420758cf681dd493dd8fcf65dc7cf32f512caf7405426"} Jul 13 19:27:33.797695 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:33.797670 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9vgdf"] Jul 13 19:27:33.800073 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:27:33.800043 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod27d03ef3_472d_41a1_9d3a_1ccd7f9b7eca.slice/crio-90fd2059380ecb7c30b19c70d200c630df659cf454668a0c9eb79e2b97338a30 WatchSource:0}: Error finding container 90fd2059380ecb7c30b19c70d200c630df659cf454668a0c9eb79e2b97338a30: Status 404 returned error can't find the container with id 90fd2059380ecb7c30b19c70d200c630df659cf454668a0c9eb79e2b97338a30 Jul 13 19:27:34.700179 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:34.700138 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9vgdf" event={"ID":"27d03ef3-472d-41a1-9d3a-1ccd7f9b7eca","Type":"ContainerStarted","Data":"90fd2059380ecb7c30b19c70d200c630df659cf454668a0c9eb79e2b97338a30"} Jul 13 19:27:34.944260 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:34.944213 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hfdzm"] Jul 13 19:27:34.947542 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:34.947397 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfdzm" Jul 13 19:27:34.949678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:34.949651 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-7cqjg\"" Jul 13 19:27:34.963346 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:34.963312 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hfdzm"] Jul 13 19:27:35.144719 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:35.144679 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99047ba3-4aa9-474f-8e6d-5c85f7bb8fbd-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hfdzm\" (UID: \"99047ba3-4aa9-474f-8e6d-5c85f7bb8fbd\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfdzm" Jul 13 19:27:35.144921 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:35.144728 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m266k\" (UniqueName: \"kubernetes.io/projected/99047ba3-4aa9-474f-8e6d-5c85f7bb8fbd-kube-api-access-m266k\") pod \"cert-manager-cainjector-559cdd8c87-hfdzm\" (UID: \"99047ba3-4aa9-474f-8e6d-5c85f7bb8fbd\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfdzm" Jul 13 19:27:35.245681 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:35.245587 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99047ba3-4aa9-474f-8e6d-5c85f7bb8fbd-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hfdzm\" (UID: \"99047ba3-4aa9-474f-8e6d-5c85f7bb8fbd\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfdzm" Jul 13 19:27:35.245681 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:35.245644 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m266k\" (UniqueName: \"kubernetes.io/projected/99047ba3-4aa9-474f-8e6d-5c85f7bb8fbd-kube-api-access-m266k\") pod \"cert-manager-cainjector-559cdd8c87-hfdzm\" (UID: \"99047ba3-4aa9-474f-8e6d-5c85f7bb8fbd\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfdzm" Jul 13 19:27:35.254946 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:35.254912 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99047ba3-4aa9-474f-8e6d-5c85f7bb8fbd-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hfdzm\" (UID: \"99047ba3-4aa9-474f-8e6d-5c85f7bb8fbd\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfdzm" Jul 13 19:27:35.255128 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:35.255026 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m266k\" (UniqueName: \"kubernetes.io/projected/99047ba3-4aa9-474f-8e6d-5c85f7bb8fbd-kube-api-access-m266k\") pod \"cert-manager-cainjector-559cdd8c87-hfdzm\" (UID: \"99047ba3-4aa9-474f-8e6d-5c85f7bb8fbd\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfdzm" Jul 13 19:27:35.260725 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:35.260694 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfdzm" Jul 13 19:27:35.930822 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:35.930785 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hfdzm"] Jul 13 19:27:36.714333 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:36.714167 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" event={"ID":"1156b07e-2f94-4625-81c9-e9c0647d17e0","Type":"ContainerStarted","Data":"e92d77deea088ded8c07dedbae4dd85593cd9a4300efb5d3c7c272d22f895d23"} Jul 13 19:27:36.776899 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:27:36.776852 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99047ba3_4aa9_474f_8e6d_5c85f7bb8fbd.slice/crio-e15b07a26671be58290eab8a22980706e3e7be83a65a4a69c454763322149190 WatchSource:0}: Error finding container e15b07a26671be58290eab8a22980706e3e7be83a65a4a69c454763322149190: Status 404 returned error can't find the container with id e15b07a26671be58290eab8a22980706e3e7be83a65a4a69c454763322149190 Jul 13 19:27:37.720401 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:37.720362 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9vgdf" event={"ID":"27d03ef3-472d-41a1-9d3a-1ccd7f9b7eca","Type":"ContainerStarted","Data":"d2161d4888345fcd0819128e2ff04db88117f3b88c11421a76819a36609e6e0c"} Jul 13 19:27:37.720899 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:37.720420 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9vgdf" Jul 13 19:27:37.722486 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:37.722460 2556 generic.go:356] "Generic (PLEG): container finished" podID="1156b07e-2f94-4625-81c9-e9c0647d17e0" containerID="e92d77deea088ded8c07dedbae4dd85593cd9a4300efb5d3c7c272d22f895d23" exitCode=0 Jul 13 19:27:37.722616 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:37.722528 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" event={"ID":"1156b07e-2f94-4625-81c9-e9c0647d17e0","Type":"ContainerDied","Data":"e92d77deea088ded8c07dedbae4dd85593cd9a4300efb5d3c7c272d22f895d23"} Jul 13 19:27:37.724349 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:37.724313 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfdzm" event={"ID":"99047ba3-4aa9-474f-8e6d-5c85f7bb8fbd","Type":"ContainerStarted","Data":"0edf632a53155f9874074ff9477173773ca0445edff73bf7932fc1eb75fe08b0"} Jul 13 19:27:37.724483 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:37.724357 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfdzm" event={"ID":"99047ba3-4aa9-474f-8e6d-5c85f7bb8fbd","Type":"ContainerStarted","Data":"e15b07a26671be58290eab8a22980706e3e7be83a65a4a69c454763322149190"} Jul 13 19:27:37.740016 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:37.739950 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-9vgdf" podStartSLOduration=1.7266198579999998 podStartE2EDuration="4.739926748s" totalImagesPullingTime="3.01330689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:27:37.738331053 +0000 UTC m=+352.852238765" watchObservedRunningTime="2026-07-13 19:27:37.739926748 +0000 UTC m=+352.853834458" Jul 13 19:27:37.755453 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:37.755325 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfdzm" podStartSLOduration=3.287288756 podStartE2EDuration="3.755303044s" totalImagesPullingTime="468.014288ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:27:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:27:37.755004363 +0000 UTC m=+352.868912115" watchObservedRunningTime="2026-07-13 19:27:37.755303044 +0000 UTC m=+352.869210758" Jul 13 19:27:38.731732 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:38.731694 2556 generic.go:356] "Generic (PLEG): container finished" podID="1156b07e-2f94-4625-81c9-e9c0647d17e0" containerID="efdf32943ad4618a5e7c6632057e5ed4e7674db4df7de66572c4b985a4ea63e6" exitCode=0 Jul 13 19:27:38.732252 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:38.731799 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" event={"ID":"1156b07e-2f94-4625-81c9-e9c0647d17e0","Type":"ContainerDied","Data":"efdf32943ad4618a5e7c6632057e5ed4e7674db4df7de66572c4b985a4ea63e6"} Jul 13 19:27:39.878163 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:39.878137 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" Jul 13 19:27:39.886354 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:39.886326 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1156b07e-2f94-4625-81c9-e9c0647d17e0-util\") pod \"1156b07e-2f94-4625-81c9-e9c0647d17e0\" (UID: \"1156b07e-2f94-4625-81c9-e9c0647d17e0\") " Jul 13 19:27:39.886465 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:39.886394 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69ml5\" (UniqueName: \"kubernetes.io/projected/1156b07e-2f94-4625-81c9-e9c0647d17e0-kube-api-access-69ml5\") pod \"1156b07e-2f94-4625-81c9-e9c0647d17e0\" (UID: \"1156b07e-2f94-4625-81c9-e9c0647d17e0\") " Jul 13 19:27:39.886465 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:39.886421 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1156b07e-2f94-4625-81c9-e9c0647d17e0-bundle\") pod \"1156b07e-2f94-4625-81c9-e9c0647d17e0\" (UID: \"1156b07e-2f94-4625-81c9-e9c0647d17e0\") " Jul 13 19:27:39.886971 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:39.886936 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1156b07e-2f94-4625-81c9-e9c0647d17e0-bundle" (OuterVolumeSpecName: "bundle") pod "1156b07e-2f94-4625-81c9-e9c0647d17e0" (UID: "1156b07e-2f94-4625-81c9-e9c0647d17e0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:27:39.888580 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:39.888555 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1156b07e-2f94-4625-81c9-e9c0647d17e0-kube-api-access-69ml5" (OuterVolumeSpecName: "kube-api-access-69ml5") pod "1156b07e-2f94-4625-81c9-e9c0647d17e0" (UID: "1156b07e-2f94-4625-81c9-e9c0647d17e0"). InnerVolumeSpecName "kube-api-access-69ml5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:27:39.892196 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:39.892154 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1156b07e-2f94-4625-81c9-e9c0647d17e0-util" (OuterVolumeSpecName: "util") pod "1156b07e-2f94-4625-81c9-e9c0647d17e0" (UID: "1156b07e-2f94-4625-81c9-e9c0647d17e0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:27:39.987106 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:39.987059 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1156b07e-2f94-4625-81c9-e9c0647d17e0-util\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:27:39.987106 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:39.987098 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69ml5\" (UniqueName: \"kubernetes.io/projected/1156b07e-2f94-4625-81c9-e9c0647d17e0-kube-api-access-69ml5\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:27:39.987106 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:39.987108 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1156b07e-2f94-4625-81c9-e9c0647d17e0-bundle\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:27:40.744085 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:40.744051 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" Jul 13 19:27:40.744241 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:40.744083 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw425z" event={"ID":"1156b07e-2f94-4625-81c9-e9c0647d17e0","Type":"ContainerDied","Data":"36cbf2ab7e4ead1dd02c1436dc0fa16665d5b42515cc7eaad802196ae9eb80f0"} Jul 13 19:27:40.744241 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:40.744132 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36cbf2ab7e4ead1dd02c1436dc0fa16665d5b42515cc7eaad802196ae9eb80f0" Jul 13 19:27:43.734915 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:43.734871 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9vgdf" Jul 13 19:27:45.707448 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.707416 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dc2nt"] Jul 13 19:27:45.707845 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.707829 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1156b07e-2f94-4625-81c9-e9c0647d17e0" containerName="util" Jul 13 19:27:45.707845 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.707846 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1156b07e-2f94-4625-81c9-e9c0647d17e0" containerName="util" Jul 13 19:27:45.707939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.707861 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1156b07e-2f94-4625-81c9-e9c0647d17e0" containerName="pull" Jul 13 19:27:45.707939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.707867 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1156b07e-2f94-4625-81c9-e9c0647d17e0" containerName="pull" Jul 13 19:27:45.707939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.707874 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1156b07e-2f94-4625-81c9-e9c0647d17e0" containerName="extract" Jul 13 19:27:45.707939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.707880 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1156b07e-2f94-4625-81c9-e9c0647d17e0" containerName="extract" Jul 13 19:27:45.707939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.707936 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1156b07e-2f94-4625-81c9-e9c0647d17e0" containerName="extract" Jul 13 19:27:45.714937 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.714910 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dc2nt" Jul 13 19:27:45.718484 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.717613 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-27crq\"" Jul 13 19:27:45.718484 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.718037 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 13 19:27:45.718484 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.718317 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 13 19:27:45.720213 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.720121 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dc2nt"] Jul 13 19:27:45.739698 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.739651 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nrtk\" (UniqueName: \"kubernetes.io/projected/34b34ab1-edb9-4274-87be-0b2e174e2547-kube-api-access-7nrtk\") pod \"openshift-lws-operator-fd8ccff4c-dc2nt\" (UID: \"34b34ab1-edb9-4274-87be-0b2e174e2547\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dc2nt" Jul 13 19:27:45.740048 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.740020 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34b34ab1-edb9-4274-87be-0b2e174e2547-tmp\") pod \"openshift-lws-operator-fd8ccff4c-dc2nt\" (UID: \"34b34ab1-edb9-4274-87be-0b2e174e2547\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dc2nt" Jul 13 19:27:45.842646 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.842600 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34b34ab1-edb9-4274-87be-0b2e174e2547-tmp\") pod \"openshift-lws-operator-fd8ccff4c-dc2nt\" (UID: \"34b34ab1-edb9-4274-87be-0b2e174e2547\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dc2nt" Jul 13 19:27:45.843208 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.842728 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nrtk\" (UniqueName: \"kubernetes.io/projected/34b34ab1-edb9-4274-87be-0b2e174e2547-kube-api-access-7nrtk\") pod \"openshift-lws-operator-fd8ccff4c-dc2nt\" (UID: \"34b34ab1-edb9-4274-87be-0b2e174e2547\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dc2nt" Jul 13 19:27:45.843766 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.843726 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34b34ab1-edb9-4274-87be-0b2e174e2547-tmp\") pod \"openshift-lws-operator-fd8ccff4c-dc2nt\" (UID: \"34b34ab1-edb9-4274-87be-0b2e174e2547\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dc2nt" Jul 13 19:27:45.851096 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.851069 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 13 19:27:45.861316 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.861277 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 13 19:27:45.871970 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:45.871936 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nrtk\" (UniqueName: \"kubernetes.io/projected/34b34ab1-edb9-4274-87be-0b2e174e2547-kube-api-access-7nrtk\") pod \"openshift-lws-operator-fd8ccff4c-dc2nt\" (UID: \"34b34ab1-edb9-4274-87be-0b2e174e2547\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dc2nt" Jul 13 19:27:46.030432 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:46.030346 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-27crq\"" Jul 13 19:27:46.039014 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:46.038984 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dc2nt" Jul 13 19:27:46.203055 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:46.203007 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dc2nt"] Jul 13 19:27:46.209848 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:27:46.209811 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34b34ab1_edb9_4274_87be_0b2e174e2547.slice/crio-0fe01d0514b2a357c3a9347fbedb38ba7a69a918160f8612805c34bef1e49cf3 WatchSource:0}: Error finding container 0fe01d0514b2a357c3a9347fbedb38ba7a69a918160f8612805c34bef1e49cf3: Status 404 returned error can't find the container with id 0fe01d0514b2a357c3a9347fbedb38ba7a69a918160f8612805c34bef1e49cf3 Jul 13 19:27:46.775154 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:46.775118 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dc2nt" event={"ID":"34b34ab1-edb9-4274-87be-0b2e174e2547","Type":"ContainerStarted","Data":"0fe01d0514b2a357c3a9347fbedb38ba7a69a918160f8612805c34bef1e49cf3"} Jul 13 19:27:48.788181 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:48.788144 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dc2nt" event={"ID":"34b34ab1-edb9-4274-87be-0b2e174e2547","Type":"ContainerStarted","Data":"7dbe9af0af353cec49e15304606d3e81a7b921d58044fb2a19cc5a36921cfab6"} Jul 13 19:27:48.806478 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:48.806416 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-dc2nt" podStartSLOduration=1.538966311 podStartE2EDuration="3.806396128s" totalImagesPullingTime="2.267429817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:27:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:27:48.804468741 +0000 UTC m=+363.918376454" watchObservedRunningTime="2026-07-13 19:27:48.806396128 +0000 UTC m=+363.920303836" Jul 13 19:27:51.680435 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:51.680399 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l"] Jul 13 19:27:51.683107 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:51.683089 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l" Jul 13 19:27:51.688928 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:51.688902 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 19:27:51.689385 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:51.689369 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 19:27:51.689476 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:51.689412 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rkn6h\"" Jul 13 19:27:51.704874 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:51.704827 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l"] Jul 13 19:27:51.798618 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:51.798580 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2c4ba65-2cba-4fdb-b823-cc660863c0fe-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l\" (UID: \"e2c4ba65-2cba-4fdb-b823-cc660863c0fe\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l" Jul 13 19:27:51.798855 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:51.798668 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2c4ba65-2cba-4fdb-b823-cc660863c0fe-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l\" (UID: \"e2c4ba65-2cba-4fdb-b823-cc660863c0fe\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l" Jul 13 19:27:51.798855 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:51.798700 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bzlq\" (UniqueName: \"kubernetes.io/projected/e2c4ba65-2cba-4fdb-b823-cc660863c0fe-kube-api-access-8bzlq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l\" (UID: \"e2c4ba65-2cba-4fdb-b823-cc660863c0fe\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l" Jul 13 19:27:51.899611 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:51.899573 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2c4ba65-2cba-4fdb-b823-cc660863c0fe-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l\" (UID: \"e2c4ba65-2cba-4fdb-b823-cc660863c0fe\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l" Jul 13 19:27:51.899858 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:51.899671 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2c4ba65-2cba-4fdb-b823-cc660863c0fe-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l\" (UID: \"e2c4ba65-2cba-4fdb-b823-cc660863c0fe\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l" Jul 13 19:27:51.899858 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:51.899701 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bzlq\" (UniqueName: \"kubernetes.io/projected/e2c4ba65-2cba-4fdb-b823-cc660863c0fe-kube-api-access-8bzlq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l\" (UID: \"e2c4ba65-2cba-4fdb-b823-cc660863c0fe\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l" Jul 13 19:27:51.900071 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:51.900047 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2c4ba65-2cba-4fdb-b823-cc660863c0fe-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l\" (UID: \"e2c4ba65-2cba-4fdb-b823-cc660863c0fe\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l" Jul 13 19:27:51.900163 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:51.900134 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2c4ba65-2cba-4fdb-b823-cc660863c0fe-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l\" (UID: \"e2c4ba65-2cba-4fdb-b823-cc660863c0fe\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l" Jul 13 19:27:51.909402 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:51.909373 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bzlq\" (UniqueName: \"kubernetes.io/projected/e2c4ba65-2cba-4fdb-b823-cc660863c0fe-kube-api-access-8bzlq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l\" (UID: \"e2c4ba65-2cba-4fdb-b823-cc660863c0fe\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l" Jul 13 19:27:51.993133 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:51.993018 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l" Jul 13 19:27:52.130133 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:52.130107 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l"] Jul 13 19:27:52.132479 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:27:52.132443 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2c4ba65_2cba_4fdb_b823_cc660863c0fe.slice/crio-5eb6452fbca051c4d4c634008130ceec20421e7622b9583e30cbb20471adb4df WatchSource:0}: Error finding container 5eb6452fbca051c4d4c634008130ceec20421e7622b9583e30cbb20471adb4df: Status 404 returned error can't find the container with id 5eb6452fbca051c4d4c634008130ceec20421e7622b9583e30cbb20471adb4df Jul 13 19:27:52.812718 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:52.812677 2556 generic.go:356] "Generic (PLEG): container finished" podID="e2c4ba65-2cba-4fdb-b823-cc660863c0fe" containerID="3f58ffbfd8cb1d838cfd0d37603a6dc1fd7ec53741dbb38b5a31675541a21bd5" exitCode=0 Jul 13 19:27:52.813154 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:52.812763 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l" event={"ID":"e2c4ba65-2cba-4fdb-b823-cc660863c0fe","Type":"ContainerDied","Data":"3f58ffbfd8cb1d838cfd0d37603a6dc1fd7ec53741dbb38b5a31675541a21bd5"} Jul 13 19:27:52.813154 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:52.812865 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l" event={"ID":"e2c4ba65-2cba-4fdb-b823-cc660863c0fe","Type":"ContainerStarted","Data":"5eb6452fbca051c4d4c634008130ceec20421e7622b9583e30cbb20471adb4df"} Jul 13 19:27:54.824699 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:54.824658 2556 generic.go:356] "Generic (PLEG): container finished" podID="e2c4ba65-2cba-4fdb-b823-cc660863c0fe" containerID="ce1a395c8dacd1ecc197ed059f071c2e335d89c9025f0cf77cc5d43c735427a2" exitCode=0 Jul 13 19:27:54.825148 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:54.824737 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l" event={"ID":"e2c4ba65-2cba-4fdb-b823-cc660863c0fe","Type":"ContainerDied","Data":"ce1a395c8dacd1ecc197ed059f071c2e335d89c9025f0cf77cc5d43c735427a2"} Jul 13 19:27:55.832345 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:55.832299 2556 generic.go:356] "Generic (PLEG): container finished" podID="e2c4ba65-2cba-4fdb-b823-cc660863c0fe" containerID="b45d5ce80edf8dd84368fc975539f717650a40a4a062813998720052ebc404f0" exitCode=0 Jul 13 19:27:55.832748 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:55.832386 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l" event={"ID":"e2c4ba65-2cba-4fdb-b823-cc660863c0fe","Type":"ContainerDied","Data":"b45d5ce80edf8dd84368fc975539f717650a40a4a062813998720052ebc404f0"} Jul 13 19:27:56.980353 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:56.980325 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l" Jul 13 19:27:57.047856 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:57.047818 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bzlq\" (UniqueName: \"kubernetes.io/projected/e2c4ba65-2cba-4fdb-b823-cc660863c0fe-kube-api-access-8bzlq\") pod \"e2c4ba65-2cba-4fdb-b823-cc660863c0fe\" (UID: \"e2c4ba65-2cba-4fdb-b823-cc660863c0fe\") " Jul 13 19:27:57.048031 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:57.047938 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2c4ba65-2cba-4fdb-b823-cc660863c0fe-bundle\") pod \"e2c4ba65-2cba-4fdb-b823-cc660863c0fe\" (UID: \"e2c4ba65-2cba-4fdb-b823-cc660863c0fe\") " Jul 13 19:27:57.048031 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:57.047959 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2c4ba65-2cba-4fdb-b823-cc660863c0fe-util\") pod \"e2c4ba65-2cba-4fdb-b823-cc660863c0fe\" (UID: \"e2c4ba65-2cba-4fdb-b823-cc660863c0fe\") " Jul 13 19:27:57.048685 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:57.048645 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2c4ba65-2cba-4fdb-b823-cc660863c0fe-bundle" (OuterVolumeSpecName: "bundle") pod "e2c4ba65-2cba-4fdb-b823-cc660863c0fe" (UID: "e2c4ba65-2cba-4fdb-b823-cc660863c0fe"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:27:57.050083 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:57.050054 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2c4ba65-2cba-4fdb-b823-cc660863c0fe-kube-api-access-8bzlq" (OuterVolumeSpecName: "kube-api-access-8bzlq") pod "e2c4ba65-2cba-4fdb-b823-cc660863c0fe" (UID: "e2c4ba65-2cba-4fdb-b823-cc660863c0fe"). InnerVolumeSpecName "kube-api-access-8bzlq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:27:57.053826 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:57.053801 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2c4ba65-2cba-4fdb-b823-cc660863c0fe-util" (OuterVolumeSpecName: "util") pod "e2c4ba65-2cba-4fdb-b823-cc660863c0fe" (UID: "e2c4ba65-2cba-4fdb-b823-cc660863c0fe"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:27:57.149455 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:57.149361 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2c4ba65-2cba-4fdb-b823-cc660863c0fe-bundle\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:27:57.149455 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:57.149395 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2c4ba65-2cba-4fdb-b823-cc660863c0fe-util\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:27:57.149455 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:57.149405 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bzlq\" (UniqueName: \"kubernetes.io/projected/e2c4ba65-2cba-4fdb-b823-cc660863c0fe-kube-api-access-8bzlq\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:27:57.844380 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:57.844340 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l" event={"ID":"e2c4ba65-2cba-4fdb-b823-cc660863c0fe","Type":"ContainerDied","Data":"5eb6452fbca051c4d4c634008130ceec20421e7622b9583e30cbb20471adb4df"} Jul 13 19:27:57.844531 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:57.844397 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5eb6452fbca051c4d4c634008130ceec20421e7622b9583e30cbb20471adb4df" Jul 13 19:27:57.844531 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:27:57.844351 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7sc4l" Jul 13 19:28:08.772603 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.772554 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq"] Jul 13 19:28:08.773414 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.773389 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2c4ba65-2cba-4fdb-b823-cc660863c0fe" containerName="util" Jul 13 19:28:08.773501 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.773418 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2c4ba65-2cba-4fdb-b823-cc660863c0fe" containerName="util" Jul 13 19:28:08.773501 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.773488 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2c4ba65-2cba-4fdb-b823-cc660863c0fe" containerName="pull" Jul 13 19:28:08.773501 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.773496 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2c4ba65-2cba-4fdb-b823-cc660863c0fe" containerName="pull" Jul 13 19:28:08.773636 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.773511 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2c4ba65-2cba-4fdb-b823-cc660863c0fe" containerName="extract" Jul 13 19:28:08.773636 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.773521 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2c4ba65-2cba-4fdb-b823-cc660863c0fe" containerName="extract" Jul 13 19:28:08.773726 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.773671 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2c4ba65-2cba-4fdb-b823-cc660863c0fe" containerName="extract" Jul 13 19:28:08.777328 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.777298 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:08.779932 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.779906 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 13 19:28:08.780089 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.779987 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 13 19:28:08.780299 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.780244 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7pkj5\"" Jul 13 19:28:08.780411 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.780311 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 13 19:28:08.780464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.780437 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 13 19:28:08.817554 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.817517 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq"] Jul 13 19:28:08.866333 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.866294 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xp65\" (UniqueName: \"kubernetes.io/projected/de2480a1-3f58-4a81-835f-1ee281d2ba2e-kube-api-access-5xp65\") pod \"opendatahub-operator-controller-manager-6996d85cfb-j9wjq\" (UID: \"de2480a1-3f58-4a81-835f-1ee281d2ba2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:08.866556 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.866353 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/de2480a1-3f58-4a81-835f-1ee281d2ba2e-manifests\") pod \"opendatahub-operator-controller-manager-6996d85cfb-j9wjq\" (UID: \"de2480a1-3f58-4a81-835f-1ee281d2ba2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:08.866556 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.866472 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de2480a1-3f58-4a81-835f-1ee281d2ba2e-tmp\") pod \"opendatahub-operator-controller-manager-6996d85cfb-j9wjq\" (UID: \"de2480a1-3f58-4a81-835f-1ee281d2ba2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:08.866556 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.866527 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/de2480a1-3f58-4a81-835f-1ee281d2ba2e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6996d85cfb-j9wjq\" (UID: \"de2480a1-3f58-4a81-835f-1ee281d2ba2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:08.866673 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.866574 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de2480a1-3f58-4a81-835f-1ee281d2ba2e-webhook-cert\") pod \"opendatahub-operator-controller-manager-6996d85cfb-j9wjq\" (UID: \"de2480a1-3f58-4a81-835f-1ee281d2ba2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:08.967082 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.967035 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de2480a1-3f58-4a81-835f-1ee281d2ba2e-tmp\") pod \"opendatahub-operator-controller-manager-6996d85cfb-j9wjq\" (UID: \"de2480a1-3f58-4a81-835f-1ee281d2ba2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:08.967317 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.967229 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/de2480a1-3f58-4a81-835f-1ee281d2ba2e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6996d85cfb-j9wjq\" (UID: \"de2480a1-3f58-4a81-835f-1ee281d2ba2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:08.967386 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.967346 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de2480a1-3f58-4a81-835f-1ee281d2ba2e-webhook-cert\") pod \"opendatahub-operator-controller-manager-6996d85cfb-j9wjq\" (UID: \"de2480a1-3f58-4a81-835f-1ee281d2ba2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:08.967436 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.967400 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xp65\" (UniqueName: \"kubernetes.io/projected/de2480a1-3f58-4a81-835f-1ee281d2ba2e-kube-api-access-5xp65\") pod \"opendatahub-operator-controller-manager-6996d85cfb-j9wjq\" (UID: \"de2480a1-3f58-4a81-835f-1ee281d2ba2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:08.967807 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.967507 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/de2480a1-3f58-4a81-835f-1ee281d2ba2e-manifests\") pod \"opendatahub-operator-controller-manager-6996d85cfb-j9wjq\" (UID: \"de2480a1-3f58-4a81-835f-1ee281d2ba2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:08.967944 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.967852 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/de2480a1-3f58-4a81-835f-1ee281d2ba2e-manifests\") pod \"opendatahub-operator-controller-manager-6996d85cfb-j9wjq\" (UID: \"de2480a1-3f58-4a81-835f-1ee281d2ba2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:08.969617 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.969599 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de2480a1-3f58-4a81-835f-1ee281d2ba2e-tmp\") pod \"opendatahub-operator-controller-manager-6996d85cfb-j9wjq\" (UID: \"de2480a1-3f58-4a81-835f-1ee281d2ba2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:08.969988 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.969966 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/de2480a1-3f58-4a81-835f-1ee281d2ba2e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6996d85cfb-j9wjq\" (UID: \"de2480a1-3f58-4a81-835f-1ee281d2ba2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:08.970055 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.969966 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de2480a1-3f58-4a81-835f-1ee281d2ba2e-webhook-cert\") pod \"opendatahub-operator-controller-manager-6996d85cfb-j9wjq\" (UID: \"de2480a1-3f58-4a81-835f-1ee281d2ba2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:08.979146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:08.979116 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xp65\" (UniqueName: \"kubernetes.io/projected/de2480a1-3f58-4a81-835f-1ee281d2ba2e-kube-api-access-5xp65\") pod \"opendatahub-operator-controller-manager-6996d85cfb-j9wjq\" (UID: \"de2480a1-3f58-4a81-835f-1ee281d2ba2e\") " pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:09.089401 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:09.089364 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:09.247162 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:09.247123 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq"] Jul 13 19:28:09.251486 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:28:09.251452 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde2480a1_3f58_4a81_835f_1ee281d2ba2e.slice/crio-dd87d9b0896c64fa7bb783c1946ad56e43bbcb08753238838055968fc9ed18bc WatchSource:0}: Error finding container dd87d9b0896c64fa7bb783c1946ad56e43bbcb08753238838055968fc9ed18bc: Status 404 returned error can't find the container with id dd87d9b0896c64fa7bb783c1946ad56e43bbcb08753238838055968fc9ed18bc Jul 13 19:28:09.913526 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:09.913485 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" event={"ID":"de2480a1-3f58-4a81-835f-1ee281d2ba2e","Type":"ContainerStarted","Data":"dd87d9b0896c64fa7bb783c1946ad56e43bbcb08753238838055968fc9ed18bc"} Jul 13 19:28:12.933541 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:12.933503 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" event={"ID":"de2480a1-3f58-4a81-835f-1ee281d2ba2e","Type":"ContainerStarted","Data":"41829ce83bdaf877d91aef3acd53ed1ccb7d01e4d38741463ea616718b0dc5eb"} Jul 13 19:28:13.939732 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:13.939639 2556 generic.go:356] "Generic (PLEG): container finished" podID="de2480a1-3f58-4a81-835f-1ee281d2ba2e" containerID="41829ce83bdaf877d91aef3acd53ed1ccb7d01e4d38741463ea616718b0dc5eb" exitCode=0 Jul 13 19:28:13.940130 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:13.939723 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" event={"ID":"de2480a1-3f58-4a81-835f-1ee281d2ba2e","Type":"ContainerDied","Data":"41829ce83bdaf877d91aef3acd53ed1ccb7d01e4d38741463ea616718b0dc5eb"} Jul 13 19:28:14.947242 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:14.947198 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" event={"ID":"de2480a1-3f58-4a81-835f-1ee281d2ba2e","Type":"ContainerStarted","Data":"89bb1d31fb88c7b858f848de8445313d078d4ea79cd97914c4e2862545c184cb"} Jul 13 19:28:14.947740 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:14.947355 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:14.977481 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:14.977421 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" podStartSLOduration=2.216981696 podStartE2EDuration="6.977401927s" totalImagesPullingTime="3.633813546s" totalInitContainerRuntime="1.126606685s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:28:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:28:14.972104118 +0000 UTC m=+390.086011826" watchObservedRunningTime="2026-07-13 19:28:14.977401927 +0000 UTC m=+390.091309635" Jul 13 19:28:25.954380 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:25.954347 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6996d85cfb-j9wjq" Jul 13 19:28:38.781402 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.781358 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6"] Jul 13 19:28:38.784457 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.784433 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.786883 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.786857 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-w96v2\"" Jul 13 19:28:38.787048 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.786867 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 13 19:28:38.787127 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.787046 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 13 19:28:38.787400 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.787189 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 13 19:28:38.803989 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.803956 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6"] Jul 13 19:28:38.847356 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.847314 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.847543 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.847361 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.847543 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.847402 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.847543 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.847425 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.847543 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.847525 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.847710 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.847577 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.847710 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.847641 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.847710 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.847683 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f6df350-4c26-45b6-94bf-113187ab8c7a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.847855 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.847752 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phfz5\" (UniqueName: \"kubernetes.io/projected/6f6df350-4c26-45b6-94bf-113187ab8c7a-kube-api-access-phfz5\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.948984 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.948941 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.948984 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.948989 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.949253 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.949023 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.949253 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.949047 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.949253 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.949090 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.949253 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.949128 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.949253 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.949162 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.949520 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.949226 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f6df350-4c26-45b6-94bf-113187ab8c7a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.949520 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.949435 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.949520 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.949454 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phfz5\" (UniqueName: \"kubernetes.io/projected/6f6df350-4c26-45b6-94bf-113187ab8c7a-kube-api-access-phfz5\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.949678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.949615 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.949678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.949651 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.949809 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.949720 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.950159 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.950137 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f6df350-4c26-45b6-94bf-113187ab8c7a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.951653 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.951624 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.951857 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.951840 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.966061 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.966026 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phfz5\" (UniqueName: \"kubernetes.io/projected/6f6df350-4c26-45b6-94bf-113187ab8c7a-kube-api-access-phfz5\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:38.966198 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:38.966035 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:39.098418 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.098381 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:39.110341 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.110285 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc"] Jul 13 19:28:39.113654 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.113624 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.147166 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.147128 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc"] Jul 13 19:28:39.252707 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.252676 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.252707 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.252720 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.252946 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.252751 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.252946 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.252860 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwthg\" (UniqueName: \"kubernetes.io/projected/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-kube-api-access-jwthg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.252946 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.252907 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.253080 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.252951 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.253080 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.253010 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.253171 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.253073 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.253171 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.253122 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.273623 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.273595 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6"] Jul 13 19:28:39.275910 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:28:39.275883 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f6df350_4c26_45b6_94bf_113187ab8c7a.slice/crio-16920e38117a94e3b82967a3ad1bea80d22d472340d8be82e7950baab51b9243 WatchSource:0}: Error finding container 16920e38117a94e3b82967a3ad1bea80d22d472340d8be82e7950baab51b9243: Status 404 returned error can't find the container with id 16920e38117a94e3b82967a3ad1bea80d22d472340d8be82e7950baab51b9243 Jul 13 19:28:39.353864 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.353720 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.353864 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.353833 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.353864 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.353860 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.354146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.353879 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.354146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.353903 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwthg\" (UniqueName: \"kubernetes.io/projected/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-kube-api-access-jwthg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.354146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.353930 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.354146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.353978 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.354146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.354006 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.354146 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.354046 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.354444 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.354359 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.354444 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.354428 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.354523 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.354465 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.354751 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.354727 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.354871 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.354769 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.356154 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.356133 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.356538 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.356523 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.363220 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.363188 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.363377 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.363266 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwthg\" (UniqueName: \"kubernetes.io/projected/a890d805-f6dc-483b-aaaa-8e6bfe48ef08-kube-api-access-jwthg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mc4gc\" (UID: \"a890d805-f6dc-483b-aaaa-8e6bfe48ef08\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.432998 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.432964 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:39.614168 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:39.613936 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc"] Jul 13 19:28:39.617900 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:28:39.617866 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda890d805_f6dc_483b_aaaa_8e6bfe48ef08.slice/crio-559f7cce8ad434b99f01f6e0ea78d300ede09166ab86b83271856158ae5bb4f2 WatchSource:0}: Error finding container 559f7cce8ad434b99f01f6e0ea78d300ede09166ab86b83271856158ae5bb4f2: Status 404 returned error can't find the container with id 559f7cce8ad434b99f01f6e0ea78d300ede09166ab86b83271856158ae5bb4f2 Jul 13 19:28:40.074441 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:40.074393 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" event={"ID":"a890d805-f6dc-483b-aaaa-8e6bfe48ef08","Type":"ContainerStarted","Data":"559f7cce8ad434b99f01f6e0ea78d300ede09166ab86b83271856158ae5bb4f2"} Jul 13 19:28:40.076307 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:40.076236 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" event={"ID":"6f6df350-4c26-45b6-94bf-113187ab8c7a","Type":"ContainerStarted","Data":"16920e38117a94e3b82967a3ad1bea80d22d472340d8be82e7950baab51b9243"} Jul 13 19:28:40.968055 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:40.968018 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl"] Jul 13 19:28:40.975640 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:40.974965 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" Jul 13 19:28:40.978976 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:40.978944 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2qgqg\"" Jul 13 19:28:40.978976 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:40.978965 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 13 19:28:40.979428 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:40.979406 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 13 19:28:40.979615 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:40.979594 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 13 19:28:40.996890 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:40.996855 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl"] Jul 13 19:28:41.078367 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:41.078327 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/03b0ad2d-3e65-4b23-9a28-20ee30c350cf-cert\") pod \"lws-controller-manager-655465cc8d-q5kzl\" (UID: \"03b0ad2d-3e65-4b23-9a28-20ee30c350cf\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" Jul 13 19:28:41.078863 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:41.078374 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/03b0ad2d-3e65-4b23-9a28-20ee30c350cf-metrics-cert\") pod \"lws-controller-manager-655465cc8d-q5kzl\" (UID: \"03b0ad2d-3e65-4b23-9a28-20ee30c350cf\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" Jul 13 19:28:41.078863 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:41.078461 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/03b0ad2d-3e65-4b23-9a28-20ee30c350cf-manager-config\") pod \"lws-controller-manager-655465cc8d-q5kzl\" (UID: \"03b0ad2d-3e65-4b23-9a28-20ee30c350cf\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" Jul 13 19:28:41.078863 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:41.078527 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp4fm\" (UniqueName: \"kubernetes.io/projected/03b0ad2d-3e65-4b23-9a28-20ee30c350cf-kube-api-access-cp4fm\") pod \"lws-controller-manager-655465cc8d-q5kzl\" (UID: \"03b0ad2d-3e65-4b23-9a28-20ee30c350cf\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" Jul 13 19:28:41.180020 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:41.179983 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/03b0ad2d-3e65-4b23-9a28-20ee30c350cf-cert\") pod \"lws-controller-manager-655465cc8d-q5kzl\" (UID: \"03b0ad2d-3e65-4b23-9a28-20ee30c350cf\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" Jul 13 19:28:41.180020 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:41.180026 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/03b0ad2d-3e65-4b23-9a28-20ee30c350cf-metrics-cert\") pod \"lws-controller-manager-655465cc8d-q5kzl\" (UID: \"03b0ad2d-3e65-4b23-9a28-20ee30c350cf\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" Jul 13 19:28:41.180302 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:41.180061 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/03b0ad2d-3e65-4b23-9a28-20ee30c350cf-manager-config\") pod \"lws-controller-manager-655465cc8d-q5kzl\" (UID: \"03b0ad2d-3e65-4b23-9a28-20ee30c350cf\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" Jul 13 19:28:41.180302 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:41.180102 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp4fm\" (UniqueName: \"kubernetes.io/projected/03b0ad2d-3e65-4b23-9a28-20ee30c350cf-kube-api-access-cp4fm\") pod \"lws-controller-manager-655465cc8d-q5kzl\" (UID: \"03b0ad2d-3e65-4b23-9a28-20ee30c350cf\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" Jul 13 19:28:41.181160 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:41.181129 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/03b0ad2d-3e65-4b23-9a28-20ee30c350cf-manager-config\") pod \"lws-controller-manager-655465cc8d-q5kzl\" (UID: \"03b0ad2d-3e65-4b23-9a28-20ee30c350cf\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" Jul 13 19:28:41.183092 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:41.183055 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/03b0ad2d-3e65-4b23-9a28-20ee30c350cf-metrics-cert\") pod \"lws-controller-manager-655465cc8d-q5kzl\" (UID: \"03b0ad2d-3e65-4b23-9a28-20ee30c350cf\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" Jul 13 19:28:41.183300 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:41.183241 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/03b0ad2d-3e65-4b23-9a28-20ee30c350cf-cert\") pod \"lws-controller-manager-655465cc8d-q5kzl\" (UID: \"03b0ad2d-3e65-4b23-9a28-20ee30c350cf\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" Jul 13 19:28:41.189616 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:41.189584 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp4fm\" (UniqueName: \"kubernetes.io/projected/03b0ad2d-3e65-4b23-9a28-20ee30c350cf-kube-api-access-cp4fm\") pod \"lws-controller-manager-655465cc8d-q5kzl\" (UID: \"03b0ad2d-3e65-4b23-9a28-20ee30c350cf\") " pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" Jul 13 19:28:41.288420 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:41.288379 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" Jul 13 19:28:41.475354 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:41.475145 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl"] Jul 13 19:28:42.096287 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:42.095816 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" event={"ID":"03b0ad2d-3e65-4b23-9a28-20ee30c350cf","Type":"ContainerStarted","Data":"d43028f03692f2fc315075ee6f78bb72d1c1043821d36cd3f24150346530fe14"} Jul 13 19:28:42.294423 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:42.294380 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-f5pvv"] Jul 13 19:28:42.300658 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:42.300630 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f5pvv" Jul 13 19:28:42.311733 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:42.311709 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 13 19:28:42.311890 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:42.311762 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-k72tg\"" Jul 13 19:28:42.339512 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:42.339471 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-f5pvv"] Jul 13 19:28:42.395891 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:42.395770 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vd7n\" (UniqueName: \"kubernetes.io/projected/532f7bb2-91cc-4a13-9e97-7c379750744f-kube-api-access-2vd7n\") pod \"llmisvc-controller-manager-777c6bd7d5-f5pvv\" (UID: \"532f7bb2-91cc-4a13-9e97-7c379750744f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f5pvv" Jul 13 19:28:42.396061 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:42.395995 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/532f7bb2-91cc-4a13-9e97-7c379750744f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f5pvv\" (UID: \"532f7bb2-91cc-4a13-9e97-7c379750744f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f5pvv" Jul 13 19:28:42.497581 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:42.497542 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vd7n\" (UniqueName: \"kubernetes.io/projected/532f7bb2-91cc-4a13-9e97-7c379750744f-kube-api-access-2vd7n\") pod \"llmisvc-controller-manager-777c6bd7d5-f5pvv\" (UID: \"532f7bb2-91cc-4a13-9e97-7c379750744f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f5pvv" Jul 13 19:28:42.497802 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:42.497696 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/532f7bb2-91cc-4a13-9e97-7c379750744f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f5pvv\" (UID: \"532f7bb2-91cc-4a13-9e97-7c379750744f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f5pvv" Jul 13 19:28:42.497889 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:28:42.497858 2556 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 13 19:28:42.497949 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:28:42.497939 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532f7bb2-91cc-4a13-9e97-7c379750744f-cert podName:532f7bb2-91cc-4a13-9e97-7c379750744f nodeName:}" failed. No retries permitted until 2026-07-13 19:28:42.997914029 +0000 UTC m=+418.111821737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/532f7bb2-91cc-4a13-9e97-7c379750744f-cert") pod "llmisvc-controller-manager-777c6bd7d5-f5pvv" (UID: "532f7bb2-91cc-4a13-9e97-7c379750744f") : secret "llmisvc-webhook-server-cert" not found Jul 13 19:28:42.528248 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:42.528210 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vd7n\" (UniqueName: \"kubernetes.io/projected/532f7bb2-91cc-4a13-9e97-7c379750744f-kube-api-access-2vd7n\") pod \"llmisvc-controller-manager-777c6bd7d5-f5pvv\" (UID: \"532f7bb2-91cc-4a13-9e97-7c379750744f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f5pvv" Jul 13 19:28:43.003928 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:43.003890 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/532f7bb2-91cc-4a13-9e97-7c379750744f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f5pvv\" (UID: \"532f7bb2-91cc-4a13-9e97-7c379750744f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f5pvv" Jul 13 19:28:43.006808 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:43.006766 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/532f7bb2-91cc-4a13-9e97-7c379750744f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f5pvv\" (UID: \"532f7bb2-91cc-4a13-9e97-7c379750744f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f5pvv" Jul 13 19:28:43.177835 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:43.177761 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 19:28:43.178157 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:43.177880 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 19:28:43.178157 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:43.177920 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 19:28:43.215841 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:43.215807 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f5pvv" Jul 13 19:28:43.395279 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:43.395251 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-f5pvv"] Jul 13 19:28:43.398093 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:28:43.398061 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod532f7bb2_91cc_4a13_9e97_7c379750744f.slice/crio-30973fed2ca9e954bdc9246113396144d4ccb9bb7e77a4e87bd5d964c1d3df1a WatchSource:0}: Error finding container 30973fed2ca9e954bdc9246113396144d4ccb9bb7e77a4e87bd5d964c1d3df1a: Status 404 returned error can't find the container with id 30973fed2ca9e954bdc9246113396144d4ccb9bb7e77a4e87bd5d964c1d3df1a Jul 13 19:28:44.111463 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:44.111415 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" event={"ID":"a890d805-f6dc-483b-aaaa-8e6bfe48ef08","Type":"ContainerStarted","Data":"5e6abae3214c6b40ff969fea55c8368b45cdfce20b48aeb2364e930cb02c9afc"} Jul 13 19:28:44.112998 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:44.112960 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f5pvv" event={"ID":"532f7bb2-91cc-4a13-9e97-7c379750744f","Type":"ContainerStarted","Data":"30973fed2ca9e954bdc9246113396144d4ccb9bb7e77a4e87bd5d964c1d3df1a"} Jul 13 19:28:44.134822 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:44.134750 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" podStartSLOduration=1.577333028 podStartE2EDuration="5.134728816s" totalImagesPullingTime="3.557395788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:28:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:28:44.132608806 +0000 UTC m=+419.246516533" watchObservedRunningTime="2026-07-13 19:28:44.134728816 +0000 UTC m=+419.248636594" Jul 13 19:28:44.433886 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:44.433766 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:44.438862 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:44.438833 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:45.124796 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:45.124727 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" event={"ID":"03b0ad2d-3e65-4b23-9a28-20ee30c350cf","Type":"ContainerStarted","Data":"74f871f91bcf7ab42efa12238d02319f37a76cfd6eeef78f89b730d864ab2718"} Jul 13 19:28:45.125501 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:45.125442 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:45.126347 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:45.126325 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mc4gc" Jul 13 19:28:45.145362 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:45.145170 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" podStartSLOduration=2.05918135 podStartE2EDuration="5.145150217s" totalImagesPullingTime="3.085968867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:28:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:28:45.144617528 +0000 UTC m=+420.258525239" watchObservedRunningTime="2026-07-13 19:28:45.145150217 +0000 UTC m=+420.259057931" Jul 13 19:28:45.205587 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:45.205534 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6"] Jul 13 19:28:46.131131 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:46.131092 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" Jul 13 19:28:48.145116 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:48.145064 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f5pvv" event={"ID":"532f7bb2-91cc-4a13-9e97-7c379750744f","Type":"ContainerStarted","Data":"d3185ee21a0ff885a8b2bb7b4cac4ee33e2d36f45719341cfb4d24e674956100"} Jul 13 19:28:48.145592 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:48.145199 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f5pvv" Jul 13 19:28:52.199102 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:52.199062 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 19:28:52.199426 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:52.199157 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 19:28:52.199426 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:52.199191 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 19:28:53.173204 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:53.173163 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" event={"ID":"6f6df350-4c26-45b6-94bf-113187ab8c7a","Type":"ContainerStarted","Data":"33fb211db1683140241c58766163ade5ae8c015c589a57282a4d2b23dbaa9aa9"} Jul 13 19:28:53.173392 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:53.173237 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" podUID="6f6df350-4c26-45b6-94bf-113187ab8c7a" containerName="istio-proxy" containerID="cri-o://33fb211db1683140241c58766163ade5ae8c015c589a57282a4d2b23dbaa9aa9" gracePeriod=30 Jul 13 19:28:53.194745 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:53.194672 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f5pvv" podStartSLOduration=7.080099203 podStartE2EDuration="11.194621123s" totalImagesPullingTime="4.11452192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:28:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:28:48.187433236 +0000 UTC m=+423.301340944" watchObservedRunningTime="2026-07-13 19:28:53.194621123 +0000 UTC m=+428.308528831" Jul 13 19:28:53.195309 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:53.195277 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" podStartSLOduration=2.274477838 podStartE2EDuration="15.195263762s" totalImagesPullingTime="12.920785924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:28:53.193402022 +0000 UTC m=+428.307309750" watchObservedRunningTime="2026-07-13 19:28:53.195263762 +0000 UTC m=+428.309171569" Jul 13 19:28:54.099530 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:54.099485 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:57.139568 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:57.139516 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-655465cc8d-q5kzl" Jul 13 19:28:58.428472 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.428444 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:58.466260 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.466173 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-credential-socket\") pod \"6f6df350-4c26-45b6-94bf-113187ab8c7a\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " Jul 13 19:28:58.466260 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.466239 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-workload-certs\") pod \"6f6df350-4c26-45b6-94bf-113187ab8c7a\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " Jul 13 19:28:58.466260 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.466257 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f6df350-4c26-45b6-94bf-113187ab8c7a-istiod-ca-cert\") pod \"6f6df350-4c26-45b6-94bf-113187ab8c7a\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " Jul 13 19:28:58.466507 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.466287 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-workload-socket\") pod \"6f6df350-4c26-45b6-94bf-113187ab8c7a\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " Jul 13 19:28:58.466507 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.466316 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-token\") pod \"6f6df350-4c26-45b6-94bf-113187ab8c7a\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " Jul 13 19:28:58.466507 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.466488 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-data\") pod \"6f6df350-4c26-45b6-94bf-113187ab8c7a\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " Jul 13 19:28:58.466659 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.466542 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phfz5\" (UniqueName: \"kubernetes.io/projected/6f6df350-4c26-45b6-94bf-113187ab8c7a-kube-api-access-phfz5\") pod \"6f6df350-4c26-45b6-94bf-113187ab8c7a\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " Jul 13 19:28:58.466659 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.466535 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6f6df350-4c26-45b6-94bf-113187ab8c7a" (UID: "6f6df350-4c26-45b6-94bf-113187ab8c7a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:28:58.466659 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.466588 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-envoy\") pod \"6f6df350-4c26-45b6-94bf-113187ab8c7a\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " Jul 13 19:28:58.466659 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.466614 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-podinfo\") pod \"6f6df350-4c26-45b6-94bf-113187ab8c7a\" (UID: \"6f6df350-4c26-45b6-94bf-113187ab8c7a\") " Jul 13 19:28:58.466659 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.466614 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6f6df350-4c26-45b6-94bf-113187ab8c7a" (UID: "6f6df350-4c26-45b6-94bf-113187ab8c7a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:28:58.466910 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.466675 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f6df350-4c26-45b6-94bf-113187ab8c7a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6f6df350-4c26-45b6-94bf-113187ab8c7a" (UID: "6f6df350-4c26-45b6-94bf-113187ab8c7a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:28:58.466910 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.466790 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-data" (OuterVolumeSpecName: "istio-data") pod "6f6df350-4c26-45b6-94bf-113187ab8c7a" (UID: "6f6df350-4c26-45b6-94bf-113187ab8c7a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:28:58.467005 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.466963 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6f6df350-4c26-45b6-94bf-113187ab8c7a" (UID: "6f6df350-4c26-45b6-94bf-113187ab8c7a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:28:58.467061 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.467013 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f6df350-4c26-45b6-94bf-113187ab8c7a-istiod-ca-cert\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:28:58.467061 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.467034 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-workload-socket\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:28:58.467061 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.467048 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-data\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:28:58.467160 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.467062 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-credential-socket\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:28:58.469280 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.469248 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f6df350-4c26-45b6-94bf-113187ab8c7a-kube-api-access-phfz5" (OuterVolumeSpecName: "kube-api-access-phfz5") pod "6f6df350-4c26-45b6-94bf-113187ab8c7a" (UID: "6f6df350-4c26-45b6-94bf-113187ab8c7a"). InnerVolumeSpecName "kube-api-access-phfz5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:28:58.469280 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.469268 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6f6df350-4c26-45b6-94bf-113187ab8c7a" (UID: "6f6df350-4c26-45b6-94bf-113187ab8c7a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:28:58.469430 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.469275 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6f6df350-4c26-45b6-94bf-113187ab8c7a" (UID: "6f6df350-4c26-45b6-94bf-113187ab8c7a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 19:28:58.469430 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.469295 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-token" (OuterVolumeSpecName: "istio-token") pod "6f6df350-4c26-45b6-94bf-113187ab8c7a" (UID: "6f6df350-4c26-45b6-94bf-113187ab8c7a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:28:58.567614 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.567572 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-token\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:28:58.567614 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.567607 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phfz5\" (UniqueName: \"kubernetes.io/projected/6f6df350-4c26-45b6-94bf-113187ab8c7a-kube-api-access-phfz5\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:28:58.567614 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.567619 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-envoy\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:28:58.567893 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.567627 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f6df350-4c26-45b6-94bf-113187ab8c7a-istio-podinfo\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:28:58.567893 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:58.567636 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f6df350-4c26-45b6-94bf-113187ab8c7a-workload-certs\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:28:59.207258 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:59.207162 2556 generic.go:356] "Generic (PLEG): container finished" podID="6f6df350-4c26-45b6-94bf-113187ab8c7a" containerID="33fb211db1683140241c58766163ade5ae8c015c589a57282a4d2b23dbaa9aa9" exitCode=0 Jul 13 19:28:59.207258 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:59.207218 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" event={"ID":"6f6df350-4c26-45b6-94bf-113187ab8c7a","Type":"ContainerDied","Data":"33fb211db1683140241c58766163ade5ae8c015c589a57282a4d2b23dbaa9aa9"} Jul 13 19:28:59.207258 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:59.207232 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" Jul 13 19:28:59.207463 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:59.207273 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6" event={"ID":"6f6df350-4c26-45b6-94bf-113187ab8c7a","Type":"ContainerDied","Data":"16920e38117a94e3b82967a3ad1bea80d22d472340d8be82e7950baab51b9243"} Jul 13 19:28:59.207463 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:59.207298 2556 scope.go:117] "RemoveContainer" containerID="33fb211db1683140241c58766163ade5ae8c015c589a57282a4d2b23dbaa9aa9" Jul 13 19:28:59.216706 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:59.216683 2556 scope.go:117] "RemoveContainer" containerID="33fb211db1683140241c58766163ade5ae8c015c589a57282a4d2b23dbaa9aa9" Jul 13 19:28:59.217123 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:28:59.217098 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33fb211db1683140241c58766163ade5ae8c015c589a57282a4d2b23dbaa9aa9\": container with ID starting with 33fb211db1683140241c58766163ade5ae8c015c589a57282a4d2b23dbaa9aa9 not found: ID does not exist" containerID="33fb211db1683140241c58766163ade5ae8c015c589a57282a4d2b23dbaa9aa9" Jul 13 19:28:59.217175 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:59.217135 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33fb211db1683140241c58766163ade5ae8c015c589a57282a4d2b23dbaa9aa9"} err="failed to get container status \"33fb211db1683140241c58766163ade5ae8c015c589a57282a4d2b23dbaa9aa9\": rpc error: code = NotFound desc = could not find container \"33fb211db1683140241c58766163ade5ae8c015c589a57282a4d2b23dbaa9aa9\": container with ID starting with 33fb211db1683140241c58766163ade5ae8c015c589a57282a4d2b23dbaa9aa9 not found: ID does not exist" Jul 13 19:28:59.225267 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:59.225234 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6"] Jul 13 19:28:59.232894 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:28:59.232866 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6pxq6"] Jul 13 19:29:00.850395 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:00.850359 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f6df350-4c26-45b6-94bf-113187ab8c7a" path="/var/lib/kubelet/pods/6f6df350-4c26-45b6-94bf-113187ab8c7a/volumes" Jul 13 19:29:19.153240 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:19.153162 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f5pvv" Jul 13 19:29:46.702167 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.702131 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6575d7755d-kfhxr"] Jul 13 19:29:46.702588 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.702542 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f6df350-4c26-45b6-94bf-113187ab8c7a" containerName="istio-proxy" Jul 13 19:29:46.702588 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.702555 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f6df350-4c26-45b6-94bf-113187ab8c7a" containerName="istio-proxy" Jul 13 19:29:46.702662 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.702628 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f6df350-4c26-45b6-94bf-113187ab8c7a" containerName="istio-proxy" Jul 13 19:29:46.706935 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.706910 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.728721 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.728682 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6575d7755d-kfhxr"] Jul 13 19:29:46.818338 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.818298 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c4ae04c-364a-4660-8dea-f8adf53495d7-trusted-ca-bundle\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.818543 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.818380 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0c4ae04c-364a-4660-8dea-f8adf53495d7-console-oauth-config\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.818543 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.818419 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0c4ae04c-364a-4660-8dea-f8adf53495d7-service-ca\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.818543 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.818445 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0c4ae04c-364a-4660-8dea-f8adf53495d7-console-config\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.818652 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.818537 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv98g\" (UniqueName: \"kubernetes.io/projected/0c4ae04c-364a-4660-8dea-f8adf53495d7-kube-api-access-sv98g\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.818652 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.818584 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0c4ae04c-364a-4660-8dea-f8adf53495d7-oauth-serving-cert\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.818652 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.818615 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0c4ae04c-364a-4660-8dea-f8adf53495d7-console-serving-cert\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.919696 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.919656 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0c4ae04c-364a-4660-8dea-f8adf53495d7-service-ca\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.919917 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.919719 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0c4ae04c-364a-4660-8dea-f8adf53495d7-console-config\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.919917 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.919751 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv98g\" (UniqueName: \"kubernetes.io/projected/0c4ae04c-364a-4660-8dea-f8adf53495d7-kube-api-access-sv98g\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.919917 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.919813 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0c4ae04c-364a-4660-8dea-f8adf53495d7-oauth-serving-cert\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.919917 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.919865 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0c4ae04c-364a-4660-8dea-f8adf53495d7-console-serving-cert\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.919917 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.919900 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c4ae04c-364a-4660-8dea-f8adf53495d7-trusted-ca-bundle\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.920161 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.919987 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0c4ae04c-364a-4660-8dea-f8adf53495d7-console-oauth-config\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.920594 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.920572 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0c4ae04c-364a-4660-8dea-f8adf53495d7-console-config\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.920594 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.920586 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0c4ae04c-364a-4660-8dea-f8adf53495d7-oauth-serving-cert\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.920743 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.920577 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0c4ae04c-364a-4660-8dea-f8adf53495d7-service-ca\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.920814 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.920801 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c4ae04c-364a-4660-8dea-f8adf53495d7-trusted-ca-bundle\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.922499 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.922476 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0c4ae04c-364a-4660-8dea-f8adf53495d7-console-oauth-config\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.922710 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.922691 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0c4ae04c-364a-4660-8dea-f8adf53495d7-console-serving-cert\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:46.928995 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:46.928969 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv98g\" (UniqueName: \"kubernetes.io/projected/0c4ae04c-364a-4660-8dea-f8adf53495d7-kube-api-access-sv98g\") pod \"console-6575d7755d-kfhxr\" (UID: \"0c4ae04c-364a-4660-8dea-f8adf53495d7\") " pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:47.016876 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:47.016733 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:47.179499 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:47.179467 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6575d7755d-kfhxr"] Jul 13 19:29:47.182984 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:29:47.182951 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c4ae04c_364a_4660_8dea_f8adf53495d7.slice/crio-e3cfded4d484e8553641010c8ec3fda41a84f4734f873a3725a58adc9147009e WatchSource:0}: Error finding container e3cfded4d484e8553641010c8ec3fda41a84f4734f873a3725a58adc9147009e: Status 404 returned error can't find the container with id e3cfded4d484e8553641010c8ec3fda41a84f4734f873a3725a58adc9147009e Jul 13 19:29:47.462295 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:47.462249 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6575d7755d-kfhxr" event={"ID":"0c4ae04c-364a-4660-8dea-f8adf53495d7","Type":"ContainerStarted","Data":"77c1841a4a09b5d96f6db566c9f16c4041f81c37ac340dfc206f278a50f9020c"} Jul 13 19:29:47.462501 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:47.462321 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6575d7755d-kfhxr" event={"ID":"0c4ae04c-364a-4660-8dea-f8adf53495d7","Type":"ContainerStarted","Data":"e3cfded4d484e8553641010c8ec3fda41a84f4734f873a3725a58adc9147009e"} Jul 13 19:29:47.482583 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:47.482524 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6575d7755d-kfhxr" podStartSLOduration=1.482504047 podStartE2EDuration="1.482504047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:29:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:29:47.481799423 +0000 UTC m=+482.595707129" watchObservedRunningTime="2026-07-13 19:29:47.482504047 +0000 UTC m=+482.596411829" Jul 13 19:29:50.570552 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.570514 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms"] Jul 13 19:29:50.579246 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.579199 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms" Jul 13 19:29:50.581436 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.581407 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 19:29:50.581652 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.581410 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 19:29:50.582084 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.582062 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-88br4\"" Jul 13 19:29:50.582210 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.582176 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms"] Jul 13 19:29:50.657469 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.657424 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbp42\" (UniqueName: \"kubernetes.io/projected/1a6ed951-de47-4ce7-a425-5704fdea97b6-kube-api-access-fbp42\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms\" (UID: \"1a6ed951-de47-4ce7-a425-5704fdea97b6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms" Jul 13 19:29:50.657469 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.657470 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a6ed951-de47-4ce7-a425-5704fdea97b6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms\" (UID: \"1a6ed951-de47-4ce7-a425-5704fdea97b6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms" Jul 13 19:29:50.657693 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.657567 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a6ed951-de47-4ce7-a425-5704fdea97b6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms\" (UID: \"1a6ed951-de47-4ce7-a425-5704fdea97b6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms" Jul 13 19:29:50.758959 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.758912 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbp42\" (UniqueName: \"kubernetes.io/projected/1a6ed951-de47-4ce7-a425-5704fdea97b6-kube-api-access-fbp42\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms\" (UID: \"1a6ed951-de47-4ce7-a425-5704fdea97b6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms" Jul 13 19:29:50.759124 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.758966 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a6ed951-de47-4ce7-a425-5704fdea97b6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms\" (UID: \"1a6ed951-de47-4ce7-a425-5704fdea97b6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms" Jul 13 19:29:50.759124 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.759040 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a6ed951-de47-4ce7-a425-5704fdea97b6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms\" (UID: \"1a6ed951-de47-4ce7-a425-5704fdea97b6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms" Jul 13 19:29:50.759405 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.759384 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a6ed951-de47-4ce7-a425-5704fdea97b6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms\" (UID: \"1a6ed951-de47-4ce7-a425-5704fdea97b6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms" Jul 13 19:29:50.759444 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.759421 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a6ed951-de47-4ce7-a425-5704fdea97b6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms\" (UID: \"1a6ed951-de47-4ce7-a425-5704fdea97b6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms" Jul 13 19:29:50.767153 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.767125 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbp42\" (UniqueName: \"kubernetes.io/projected/1a6ed951-de47-4ce7-a425-5704fdea97b6-kube-api-access-fbp42\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms\" (UID: \"1a6ed951-de47-4ce7-a425-5704fdea97b6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms" Jul 13 19:29:50.860508 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.860419 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn"] Jul 13 19:29:50.864589 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.864566 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn" Jul 13 19:29:50.872281 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.872250 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn"] Jul 13 19:29:50.890791 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.890746 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms" Jul 13 19:29:50.961336 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.961300 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b754ecef-c6d5-40b8-8bc8-5a0f4cee389e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn\" (UID: \"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn" Jul 13 19:29:50.961513 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.961384 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b754ecef-c6d5-40b8-8bc8-5a0f4cee389e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn\" (UID: \"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn" Jul 13 19:29:50.961513 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:50.961429 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwwj2\" (UniqueName: \"kubernetes.io/projected/b754ecef-c6d5-40b8-8bc8-5a0f4cee389e-kube-api-access-pwwj2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn\" (UID: \"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn" Jul 13 19:29:51.037141 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.037100 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms"] Jul 13 19:29:51.040391 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:29:51.040362 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a6ed951_de47_4ce7_a425_5704fdea97b6.slice/crio-ba6d5f627bb503a907137a9d6d6a8aa2bdf5057883a6f9c7992a293afdab26f0 WatchSource:0}: Error finding container ba6d5f627bb503a907137a9d6d6a8aa2bdf5057883a6f9c7992a293afdab26f0: Status 404 returned error can't find the container with id ba6d5f627bb503a907137a9d6d6a8aa2bdf5057883a6f9c7992a293afdab26f0 Jul 13 19:29:51.063216 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.063180 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b754ecef-c6d5-40b8-8bc8-5a0f4cee389e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn\" (UID: \"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn" Jul 13 19:29:51.063361 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.063267 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b754ecef-c6d5-40b8-8bc8-5a0f4cee389e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn\" (UID: \"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn" Jul 13 19:29:51.063361 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.063306 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwwj2\" (UniqueName: \"kubernetes.io/projected/b754ecef-c6d5-40b8-8bc8-5a0f4cee389e-kube-api-access-pwwj2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn\" (UID: \"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn" Jul 13 19:29:51.063549 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.063523 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b754ecef-c6d5-40b8-8bc8-5a0f4cee389e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn\" (UID: \"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn" Jul 13 19:29:51.063659 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.063639 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b754ecef-c6d5-40b8-8bc8-5a0f4cee389e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn\" (UID: \"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn" Jul 13 19:29:51.071194 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.071161 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwwj2\" (UniqueName: \"kubernetes.io/projected/b754ecef-c6d5-40b8-8bc8-5a0f4cee389e-kube-api-access-pwwj2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn\" (UID: \"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn" Jul 13 19:29:51.176491 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.176451 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn" Jul 13 19:29:51.263456 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.263402 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s"] Jul 13 19:29:51.267975 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.267945 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s" Jul 13 19:29:51.274321 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.274292 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s"] Jul 13 19:29:51.320926 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.320897 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn"] Jul 13 19:29:51.322940 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:29:51.322909 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb754ecef_c6d5_40b8_8bc8_5a0f4cee389e.slice/crio-8ad7abdbb0d5613d99fb22bd79b770d54ead7a94b2efdb5c3e31960f23968a60 WatchSource:0}: Error finding container 8ad7abdbb0d5613d99fb22bd79b770d54ead7a94b2efdb5c3e31960f23968a60: Status 404 returned error can't find the container with id 8ad7abdbb0d5613d99fb22bd79b770d54ead7a94b2efdb5c3e31960f23968a60 Jul 13 19:29:51.366910 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.366880 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s\" (UID: \"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s" Jul 13 19:29:51.367065 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.366931 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s\" (UID: \"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s" Jul 13 19:29:51.367065 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.367032 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fl69\" (UniqueName: \"kubernetes.io/projected/0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4-kube-api-access-8fl69\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s\" (UID: \"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s" Jul 13 19:29:51.468368 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.468330 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s\" (UID: \"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s" Jul 13 19:29:51.468572 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.468396 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s\" (UID: \"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s" Jul 13 19:29:51.468572 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.468467 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fl69\" (UniqueName: \"kubernetes.io/projected/0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4-kube-api-access-8fl69\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s\" (UID: \"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s" Jul 13 19:29:51.468755 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.468736 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s\" (UID: \"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s" Jul 13 19:29:51.468824 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.468755 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s\" (UID: \"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s" Jul 13 19:29:51.476587 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.476564 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fl69\" (UniqueName: \"kubernetes.io/projected/0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4-kube-api-access-8fl69\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s\" (UID: \"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s" Jul 13 19:29:51.487436 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.487401 2556 generic.go:356] "Generic (PLEG): container finished" podID="b754ecef-c6d5-40b8-8bc8-5a0f4cee389e" containerID="5e4d72a7dc569c42bf298cc5056253a03fdb63404d258f5126be24ad28f78a7f" exitCode=0 Jul 13 19:29:51.487613 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.487496 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn" event={"ID":"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e","Type":"ContainerDied","Data":"5e4d72a7dc569c42bf298cc5056253a03fdb63404d258f5126be24ad28f78a7f"} Jul 13 19:29:51.487613 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.487550 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn" event={"ID":"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e","Type":"ContainerStarted","Data":"8ad7abdbb0d5613d99fb22bd79b770d54ead7a94b2efdb5c3e31960f23968a60"} Jul 13 19:29:51.489344 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.489319 2556 generic.go:356] "Generic (PLEG): container finished" podID="1a6ed951-de47-4ce7-a425-5704fdea97b6" containerID="e234ee6e7f6998f90ddb9d869f7dc98c043294883df887660cdf9754c4c61ca5" exitCode=0 Jul 13 19:29:51.489450 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.489364 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms" event={"ID":"1a6ed951-de47-4ce7-a425-5704fdea97b6","Type":"ContainerDied","Data":"e234ee6e7f6998f90ddb9d869f7dc98c043294883df887660cdf9754c4c61ca5"} Jul 13 19:29:51.489450 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.489398 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms" event={"ID":"1a6ed951-de47-4ce7-a425-5704fdea97b6","Type":"ContainerStarted","Data":"ba6d5f627bb503a907137a9d6d6a8aa2bdf5057883a6f9c7992a293afdab26f0"} Jul 13 19:29:51.581587 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.581548 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s" Jul 13 19:29:51.662686 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.662649 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh"] Jul 13 19:29:51.667194 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.667164 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh" Jul 13 19:29:51.677979 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.676198 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh"] Jul 13 19:29:51.734619 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.734588 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s"] Jul 13 19:29:51.737545 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:29:51.737521 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a69d81d_09ea_4c30_9aff_bc5bb77f0ee4.slice/crio-4f3f00991e662d4741b0408a5a1b935cd8d98efe67db5a0e6c25d45026f208c6 WatchSource:0}: Error finding container 4f3f00991e662d4741b0408a5a1b935cd8d98efe67db5a0e6c25d45026f208c6: Status 404 returned error can't find the container with id 4f3f00991e662d4741b0408a5a1b935cd8d98efe67db5a0e6c25d45026f208c6 Jul 13 19:29:51.771369 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.771336 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5cbf56db-28a7-43d4-a29b-a1ec5ff0371d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh\" (UID: \"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh" Jul 13 19:29:51.771502 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.771477 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5cbf56db-28a7-43d4-a29b-a1ec5ff0371d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh\" (UID: \"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh" Jul 13 19:29:51.771637 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.771535 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsbzh\" (UniqueName: \"kubernetes.io/projected/5cbf56db-28a7-43d4-a29b-a1ec5ff0371d-kube-api-access-bsbzh\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh\" (UID: \"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh" Jul 13 19:29:51.872233 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.872198 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5cbf56db-28a7-43d4-a29b-a1ec5ff0371d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh\" (UID: \"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh" Jul 13 19:29:51.872372 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.872295 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5cbf56db-28a7-43d4-a29b-a1ec5ff0371d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh\" (UID: \"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh" Jul 13 19:29:51.872372 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.872328 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsbzh\" (UniqueName: \"kubernetes.io/projected/5cbf56db-28a7-43d4-a29b-a1ec5ff0371d-kube-api-access-bsbzh\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh\" (UID: \"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh" Jul 13 19:29:51.872617 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.872590 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5cbf56db-28a7-43d4-a29b-a1ec5ff0371d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh\" (UID: \"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh" Jul 13 19:29:51.872617 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.872606 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5cbf56db-28a7-43d4-a29b-a1ec5ff0371d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh\" (UID: \"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh" Jul 13 19:29:51.880522 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.880492 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsbzh\" (UniqueName: \"kubernetes.io/projected/5cbf56db-28a7-43d4-a29b-a1ec5ff0371d-kube-api-access-bsbzh\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh\" (UID: \"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh" Jul 13 19:29:51.981210 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:51.981082 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh" Jul 13 19:29:52.125074 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:52.124972 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh"] Jul 13 19:29:52.127403 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:29:52.127374 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cbf56db_28a7_43d4_a29b_a1ec5ff0371d.slice/crio-cb3ad9974cd132c11779981ac06e23f2bbf91f34740e2933ee0490e8432ce9d1 WatchSource:0}: Error finding container cb3ad9974cd132c11779981ac06e23f2bbf91f34740e2933ee0490e8432ce9d1: Status 404 returned error can't find the container with id cb3ad9974cd132c11779981ac06e23f2bbf91f34740e2933ee0490e8432ce9d1 Jul 13 19:29:52.496983 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:52.496950 2556 generic.go:356] "Generic (PLEG): container finished" podID="0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4" containerID="ca3fcb83f26f08fb98277bf0d71de9c31c65b1033738493e9906eb30e6982fbb" exitCode=0 Jul 13 19:29:52.497176 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:52.497030 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s" event={"ID":"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4","Type":"ContainerDied","Data":"ca3fcb83f26f08fb98277bf0d71de9c31c65b1033738493e9906eb30e6982fbb"} Jul 13 19:29:52.497176 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:52.497086 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s" event={"ID":"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4","Type":"ContainerStarted","Data":"4f3f00991e662d4741b0408a5a1b935cd8d98efe67db5a0e6c25d45026f208c6"} Jul 13 19:29:52.498966 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:52.498942 2556 generic.go:356] "Generic (PLEG): container finished" podID="5cbf56db-28a7-43d4-a29b-a1ec5ff0371d" containerID="0ae8dea8999dcb9f0dadf364a0d8432b724f69205f5ed863476fded48c60a903" exitCode=0 Jul 13 19:29:52.499080 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:52.499032 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh" event={"ID":"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d","Type":"ContainerDied","Data":"0ae8dea8999dcb9f0dadf364a0d8432b724f69205f5ed863476fded48c60a903"} Jul 13 19:29:52.499124 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:52.499089 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh" event={"ID":"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d","Type":"ContainerStarted","Data":"cb3ad9974cd132c11779981ac06e23f2bbf91f34740e2933ee0490e8432ce9d1"} Jul 13 19:29:53.507400 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:53.507357 2556 generic.go:356] "Generic (PLEG): container finished" podID="1a6ed951-de47-4ce7-a425-5704fdea97b6" containerID="a2a2b3881833f630e0ba9cac002b35fb94f8eec2141f2386996cc0091a7a59c8" exitCode=0 Jul 13 19:29:53.507870 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:53.507416 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms" event={"ID":"1a6ed951-de47-4ce7-a425-5704fdea97b6","Type":"ContainerDied","Data":"a2a2b3881833f630e0ba9cac002b35fb94f8eec2141f2386996cc0091a7a59c8"} Jul 13 19:29:53.509545 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:53.509520 2556 generic.go:356] "Generic (PLEG): container finished" podID="0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4" containerID="3a9028a09566fda339126a5cf630b10c7756f9178c97c7d7afcae0e09356b717" exitCode=0 Jul 13 19:29:53.509639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:53.509607 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s" event={"ID":"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4","Type":"ContainerDied","Data":"3a9028a09566fda339126a5cf630b10c7756f9178c97c7d7afcae0e09356b717"} Jul 13 19:29:53.511874 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:53.511844 2556 generic.go:356] "Generic (PLEG): container finished" podID="b754ecef-c6d5-40b8-8bc8-5a0f4cee389e" containerID="01527f533c27d47488505e9e9f243f7331592140fbd9941949cdb8af6af2cb0d" exitCode=0 Jul 13 19:29:53.511959 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:53.511895 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn" event={"ID":"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e","Type":"ContainerDied","Data":"01527f533c27d47488505e9e9f243f7331592140fbd9941949cdb8af6af2cb0d"} Jul 13 19:29:54.519063 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:54.519025 2556 generic.go:356] "Generic (PLEG): container finished" podID="1a6ed951-de47-4ce7-a425-5704fdea97b6" containerID="83d6458a4cfe8eb237127fee9cfb0d2f6ba2e7bf68c91413b7af03c80188dd5b" exitCode=0 Jul 13 19:29:54.519515 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:54.519099 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms" event={"ID":"1a6ed951-de47-4ce7-a425-5704fdea97b6","Type":"ContainerDied","Data":"83d6458a4cfe8eb237127fee9cfb0d2f6ba2e7bf68c91413b7af03c80188dd5b"} Jul 13 19:29:54.521405 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:54.521384 2556 generic.go:356] "Generic (PLEG): container finished" podID="0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4" containerID="0843122c821944eeac25176286f2e7bdea6ebad1d9a0f39f08bfd1f48934efb3" exitCode=0 Jul 13 19:29:54.521510 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:54.521459 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s" event={"ID":"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4","Type":"ContainerDied","Data":"0843122c821944eeac25176286f2e7bdea6ebad1d9a0f39f08bfd1f48934efb3"} Jul 13 19:29:54.523669 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:54.523648 2556 generic.go:356] "Generic (PLEG): container finished" podID="b754ecef-c6d5-40b8-8bc8-5a0f4cee389e" containerID="76b054a872640dce459520c8f015048d94202c83ec95e94425861cef3c9358ac" exitCode=0 Jul 13 19:29:54.523756 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:54.523688 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn" event={"ID":"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e","Type":"ContainerDied","Data":"76b054a872640dce459520c8f015048d94202c83ec95e94425861cef3c9358ac"} Jul 13 19:29:55.531478 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.531441 2556 generic.go:356] "Generic (PLEG): container finished" podID="5cbf56db-28a7-43d4-a29b-a1ec5ff0371d" containerID="1cc203fedc4aa1e7626768e519fbc456acb19232f941707bd7412446aa648afd" exitCode=0 Jul 13 19:29:55.531992 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.531539 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh" event={"ID":"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d","Type":"ContainerDied","Data":"1cc203fedc4aa1e7626768e519fbc456acb19232f941707bd7412446aa648afd"} Jul 13 19:29:55.702469 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.702429 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms" Jul 13 19:29:55.807753 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.807725 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn" Jul 13 19:29:55.810614 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.810586 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s" Jul 13 19:29:55.818375 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.818340 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a6ed951-de47-4ce7-a425-5704fdea97b6-bundle\") pod \"1a6ed951-de47-4ce7-a425-5704fdea97b6\" (UID: \"1a6ed951-de47-4ce7-a425-5704fdea97b6\") " Jul 13 19:29:55.818529 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.818465 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbp42\" (UniqueName: \"kubernetes.io/projected/1a6ed951-de47-4ce7-a425-5704fdea97b6-kube-api-access-fbp42\") pod \"1a6ed951-de47-4ce7-a425-5704fdea97b6\" (UID: \"1a6ed951-de47-4ce7-a425-5704fdea97b6\") " Jul 13 19:29:55.818529 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.818518 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a6ed951-de47-4ce7-a425-5704fdea97b6-util\") pod \"1a6ed951-de47-4ce7-a425-5704fdea97b6\" (UID: \"1a6ed951-de47-4ce7-a425-5704fdea97b6\") " Jul 13 19:29:55.818983 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.818956 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a6ed951-de47-4ce7-a425-5704fdea97b6-bundle" (OuterVolumeSpecName: "bundle") pod "1a6ed951-de47-4ce7-a425-5704fdea97b6" (UID: "1a6ed951-de47-4ce7-a425-5704fdea97b6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:29:55.820722 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.820684 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a6ed951-de47-4ce7-a425-5704fdea97b6-kube-api-access-fbp42" (OuterVolumeSpecName: "kube-api-access-fbp42") pod "1a6ed951-de47-4ce7-a425-5704fdea97b6" (UID: "1a6ed951-de47-4ce7-a425-5704fdea97b6"). InnerVolumeSpecName "kube-api-access-fbp42". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:29:55.824478 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.824456 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a6ed951-de47-4ce7-a425-5704fdea97b6-util" (OuterVolumeSpecName: "util") pod "1a6ed951-de47-4ce7-a425-5704fdea97b6" (UID: "1a6ed951-de47-4ce7-a425-5704fdea97b6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:29:55.919525 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.919489 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b754ecef-c6d5-40b8-8bc8-5a0f4cee389e-util\") pod \"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e\" (UID: \"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e\") " Jul 13 19:29:55.919706 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.919555 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b754ecef-c6d5-40b8-8bc8-5a0f4cee389e-bundle\") pod \"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e\" (UID: \"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e\") " Jul 13 19:29:55.919706 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.919583 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwwj2\" (UniqueName: \"kubernetes.io/projected/b754ecef-c6d5-40b8-8bc8-5a0f4cee389e-kube-api-access-pwwj2\") pod \"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e\" (UID: \"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e\") " Jul 13 19:29:55.919706 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.919603 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fl69\" (UniqueName: \"kubernetes.io/projected/0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4-kube-api-access-8fl69\") pod \"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4\" (UID: \"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4\") " Jul 13 19:29:55.919706 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.919647 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4-util\") pod \"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4\" (UID: \"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4\") " Jul 13 19:29:55.919958 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.919730 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4-bundle\") pod \"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4\" (UID: \"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4\") " Jul 13 19:29:55.920202 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.920129 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b754ecef-c6d5-40b8-8bc8-5a0f4cee389e-bundle" (OuterVolumeSpecName: "bundle") pod "b754ecef-c6d5-40b8-8bc8-5a0f4cee389e" (UID: "b754ecef-c6d5-40b8-8bc8-5a0f4cee389e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:29:55.920379 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.920320 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a6ed951-de47-4ce7-a425-5704fdea97b6-bundle\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:29:55.920528 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.920392 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbp42\" (UniqueName: \"kubernetes.io/projected/1a6ed951-de47-4ce7-a425-5704fdea97b6-kube-api-access-fbp42\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:29:55.920528 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.920426 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a6ed951-de47-4ce7-a425-5704fdea97b6-util\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:29:55.920633 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.920523 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4-bundle" (OuterVolumeSpecName: "bundle") pod "0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4" (UID: "0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:29:55.921947 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.921922 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b754ecef-c6d5-40b8-8bc8-5a0f4cee389e-kube-api-access-pwwj2" (OuterVolumeSpecName: "kube-api-access-pwwj2") pod "b754ecef-c6d5-40b8-8bc8-5a0f4cee389e" (UID: "b754ecef-c6d5-40b8-8bc8-5a0f4cee389e"). InnerVolumeSpecName "kube-api-access-pwwj2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:29:55.922029 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.921956 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4-kube-api-access-8fl69" (OuterVolumeSpecName: "kube-api-access-8fl69") pod "0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4" (UID: "0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4"). InnerVolumeSpecName "kube-api-access-8fl69". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:29:55.928926 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.928891 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b754ecef-c6d5-40b8-8bc8-5a0f4cee389e-util" (OuterVolumeSpecName: "util") pod "b754ecef-c6d5-40b8-8bc8-5a0f4cee389e" (UID: "b754ecef-c6d5-40b8-8bc8-5a0f4cee389e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:29:55.929017 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:55.928944 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4-util" (OuterVolumeSpecName: "util") pod "0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4" (UID: "0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:29:56.021795 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:56.021741 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b754ecef-c6d5-40b8-8bc8-5a0f4cee389e-util\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:29:56.021795 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:56.021788 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b754ecef-c6d5-40b8-8bc8-5a0f4cee389e-bundle\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:29:56.021795 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:56.021799 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwwj2\" (UniqueName: \"kubernetes.io/projected/b754ecef-c6d5-40b8-8bc8-5a0f4cee389e-kube-api-access-pwwj2\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:29:56.022115 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:56.021810 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fl69\" (UniqueName: \"kubernetes.io/projected/0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4-kube-api-access-8fl69\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:29:56.022115 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:56.021818 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4-util\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:29:56.022115 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:56.021826 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4-bundle\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:29:56.540063 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:56.540032 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn" Jul 13 19:29:56.540063 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:56.540039 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn" event={"ID":"b754ecef-c6d5-40b8-8bc8-5a0f4cee389e","Type":"ContainerDied","Data":"8ad7abdbb0d5613d99fb22bd79b770d54ead7a94b2efdb5c3e31960f23968a60"} Jul 13 19:29:56.540562 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:56.540088 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ad7abdbb0d5613d99fb22bd79b770d54ead7a94b2efdb5c3e31960f23968a60" Jul 13 19:29:56.542352 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:56.542326 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms" Jul 13 19:29:56.542523 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:56.542312 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms" event={"ID":"1a6ed951-de47-4ce7-a425-5704fdea97b6","Type":"ContainerDied","Data":"ba6d5f627bb503a907137a9d6d6a8aa2bdf5057883a6f9c7992a293afdab26f0"} Jul 13 19:29:56.542523 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:56.542451 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba6d5f627bb503a907137a9d6d6a8aa2bdf5057883a6f9c7992a293afdab26f0" Jul 13 19:29:56.544464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:56.544439 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s" Jul 13 19:29:56.544464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:56.544448 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s" event={"ID":"0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4","Type":"ContainerDied","Data":"4f3f00991e662d4741b0408a5a1b935cd8d98efe67db5a0e6c25d45026f208c6"} Jul 13 19:29:56.544663 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:56.544487 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f3f00991e662d4741b0408a5a1b935cd8d98efe67db5a0e6c25d45026f208c6" Jul 13 19:29:56.547017 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:56.546989 2556 generic.go:356] "Generic (PLEG): container finished" podID="5cbf56db-28a7-43d4-a29b-a1ec5ff0371d" containerID="562985f5a96db2da077c92f17f7a863e070fbd09c05c11bbadcc9ac73f0e4968" exitCode=0 Jul 13 19:29:56.547170 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:56.547019 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh" event={"ID":"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d","Type":"ContainerDied","Data":"562985f5a96db2da077c92f17f7a863e070fbd09c05c11bbadcc9ac73f0e4968"} Jul 13 19:29:57.017580 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:57.017483 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:57.017580 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:57.017550 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:57.022553 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:57.022526 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:57.556981 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:57.556953 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6575d7755d-kfhxr" Jul 13 19:29:57.608809 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:57.608761 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-786b65d94-fqdbs"] Jul 13 19:29:57.727207 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:57.727174 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh" Jul 13 19:29:57.839849 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:57.839719 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5cbf56db-28a7-43d4-a29b-a1ec5ff0371d-util\") pod \"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d\" (UID: \"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d\") " Jul 13 19:29:57.840026 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:57.839853 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5cbf56db-28a7-43d4-a29b-a1ec5ff0371d-bundle\") pod \"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d\" (UID: \"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d\") " Jul 13 19:29:57.840026 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:57.839900 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsbzh\" (UniqueName: \"kubernetes.io/projected/5cbf56db-28a7-43d4-a29b-a1ec5ff0371d-kube-api-access-bsbzh\") pod \"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d\" (UID: \"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d\") " Jul 13 19:29:57.846822 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:57.840834 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cbf56db-28a7-43d4-a29b-a1ec5ff0371d-bundle" (OuterVolumeSpecName: "bundle") pod "5cbf56db-28a7-43d4-a29b-a1ec5ff0371d" (UID: "5cbf56db-28a7-43d4-a29b-a1ec5ff0371d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:29:57.846822 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:57.846734 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cbf56db-28a7-43d4-a29b-a1ec5ff0371d-util" (OuterVolumeSpecName: "util") pod "5cbf56db-28a7-43d4-a29b-a1ec5ff0371d" (UID: "5cbf56db-28a7-43d4-a29b-a1ec5ff0371d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:29:57.849855 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:57.849711 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cbf56db-28a7-43d4-a29b-a1ec5ff0371d-kube-api-access-bsbzh" (OuterVolumeSpecName: "kube-api-access-bsbzh") pod "5cbf56db-28a7-43d4-a29b-a1ec5ff0371d" (UID: "5cbf56db-28a7-43d4-a29b-a1ec5ff0371d"). InnerVolumeSpecName "kube-api-access-bsbzh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:29:57.942034 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:57.941997 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5cbf56db-28a7-43d4-a29b-a1ec5ff0371d-bundle\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:29:57.942034 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:57.942026 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsbzh\" (UniqueName: \"kubernetes.io/projected/5cbf56db-28a7-43d4-a29b-a1ec5ff0371d-kube-api-access-bsbzh\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:29:57.942034 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:57.942036 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5cbf56db-28a7-43d4-a29b-a1ec5ff0371d-util\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:29:58.560115 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:58.560079 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh" Jul 13 19:29:58.560587 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:58.560105 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh" event={"ID":"5cbf56db-28a7-43d4-a29b-a1ec5ff0371d","Type":"ContainerDied","Data":"cb3ad9974cd132c11779981ac06e23f2bbf91f34740e2933ee0490e8432ce9d1"} Jul 13 19:29:58.560587 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:29:58.560164 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb3ad9974cd132c11779981ac06e23f2bbf91f34740e2933ee0490e8432ce9d1" Jul 13 19:30:07.004311 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004266 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-x9bzx"] Jul 13 19:30:07.004717 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004669 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b754ecef-c6d5-40b8-8bc8-5a0f4cee389e" containerName="util" Jul 13 19:30:07.004717 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004682 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b754ecef-c6d5-40b8-8bc8-5a0f4cee389e" containerName="util" Jul 13 19:30:07.004717 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004695 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cbf56db-28a7-43d4-a29b-a1ec5ff0371d" containerName="util" Jul 13 19:30:07.004717 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004700 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cbf56db-28a7-43d4-a29b-a1ec5ff0371d" containerName="util" Jul 13 19:30:07.004717 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004708 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a6ed951-de47-4ce7-a425-5704fdea97b6" containerName="pull" Jul 13 19:30:07.004717 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004715 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a6ed951-de47-4ce7-a425-5704fdea97b6" containerName="pull" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004726 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4" containerName="pull" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004731 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4" containerName="pull" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004736 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a6ed951-de47-4ce7-a425-5704fdea97b6" containerName="extract" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004741 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a6ed951-de47-4ce7-a425-5704fdea97b6" containerName="extract" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004749 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b754ecef-c6d5-40b8-8bc8-5a0f4cee389e" containerName="pull" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004755 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b754ecef-c6d5-40b8-8bc8-5a0f4cee389e" containerName="pull" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004767 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cbf56db-28a7-43d4-a29b-a1ec5ff0371d" containerName="extract" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004786 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cbf56db-28a7-43d4-a29b-a1ec5ff0371d" containerName="extract" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004795 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a6ed951-de47-4ce7-a425-5704fdea97b6" containerName="util" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004801 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a6ed951-de47-4ce7-a425-5704fdea97b6" containerName="util" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004806 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4" containerName="util" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004811 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4" containerName="util" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004817 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cbf56db-28a7-43d4-a29b-a1ec5ff0371d" containerName="pull" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004822 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cbf56db-28a7-43d4-a29b-a1ec5ff0371d" containerName="pull" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004830 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b754ecef-c6d5-40b8-8bc8-5a0f4cee389e" containerName="extract" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004835 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b754ecef-c6d5-40b8-8bc8-5a0f4cee389e" containerName="extract" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004846 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4" containerName="extract" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004851 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4" containerName="extract" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004910 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4" containerName="extract" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004918 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b754ecef-c6d5-40b8-8bc8-5a0f4cee389e" containerName="extract" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004925 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a6ed951-de47-4ce7-a425-5704fdea97b6" containerName="extract" Jul 13 19:30:07.004939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.004931 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cbf56db-28a7-43d4-a29b-a1ec5ff0371d" containerName="extract" Jul 13 19:30:07.007425 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.007408 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x9bzx" Jul 13 19:30:07.009653 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.009624 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 13 19:30:07.010122 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.010100 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 19:30:07.010122 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.010119 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-jzr48\"" Jul 13 19:30:07.010291 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.010100 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 19:30:07.024155 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.024121 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-x9bzx"] Jul 13 19:30:07.131581 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.131537 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvf9z\" (UniqueName: \"kubernetes.io/projected/c84f73db-a1da-4027-9a6e-bdc3cd0ad6b8-kube-api-access-wvf9z\") pod \"dns-operator-controller-manager-864dd78c4c-x9bzx\" (UID: \"c84f73db-a1da-4027-9a6e-bdc3cd0ad6b8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x9bzx" Jul 13 19:30:07.232739 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.232702 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvf9z\" (UniqueName: \"kubernetes.io/projected/c84f73db-a1da-4027-9a6e-bdc3cd0ad6b8-kube-api-access-wvf9z\") pod \"dns-operator-controller-manager-864dd78c4c-x9bzx\" (UID: \"c84f73db-a1da-4027-9a6e-bdc3cd0ad6b8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x9bzx" Jul 13 19:30:07.246721 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.246686 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvf9z\" (UniqueName: \"kubernetes.io/projected/c84f73db-a1da-4027-9a6e-bdc3cd0ad6b8-kube-api-access-wvf9z\") pod \"dns-operator-controller-manager-864dd78c4c-x9bzx\" (UID: \"c84f73db-a1da-4027-9a6e-bdc3cd0ad6b8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x9bzx" Jul 13 19:30:07.318288 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.318257 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x9bzx" Jul 13 19:30:07.465656 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.465631 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-x9bzx"] Jul 13 19:30:07.468005 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:30:07.467975 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc84f73db_a1da_4027_9a6e_bdc3cd0ad6b8.slice/crio-c8a4c8b9c1933c88d2e3e7f0f8662c663accc68cd81d9c16dc63788f042f3a50 WatchSource:0}: Error finding container c8a4c8b9c1933c88d2e3e7f0f8662c663accc68cd81d9c16dc63788f042f3a50: Status 404 returned error can't find the container with id c8a4c8b9c1933c88d2e3e7f0f8662c663accc68cd81d9c16dc63788f042f3a50 Jul 13 19:30:07.611521 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:07.611422 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x9bzx" event={"ID":"c84f73db-a1da-4027-9a6e-bdc3cd0ad6b8","Type":"ContainerStarted","Data":"c8a4c8b9c1933c88d2e3e7f0f8662c663accc68cd81d9c16dc63788f042f3a50"} Jul 13 19:30:11.640170 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:11.640131 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x9bzx" event={"ID":"c84f73db-a1da-4027-9a6e-bdc3cd0ad6b8","Type":"ContainerStarted","Data":"4c26ad0e8ee6ae34a1c2165ec3bddc0cd9735b543b99ca64300efe4f75a1af27"} Jul 13 19:30:11.640586 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:11.640336 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x9bzx" Jul 13 19:30:11.661001 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:11.660947 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x9bzx" podStartSLOduration=2.52707663 podStartE2EDuration="5.660928853s" totalImagesPullingTime="3.133852223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:30:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:30:11.658374949 +0000 UTC m=+506.772282658" watchObservedRunningTime="2026-07-13 19:30:11.660928853 +0000 UTC m=+506.774836563" Jul 13 19:30:15.389191 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:15.389150 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc"] Jul 13 19:30:15.395813 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:15.395752 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:15.398599 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:15.398571 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4vgc5\"" Jul 13 19:30:15.406954 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:15.406926 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b52e2578-874a-4d6e-a07f-650b6196ff5b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m8mqc\" (UID: \"b52e2578-874a-4d6e-a07f-650b6196ff5b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:15.407115 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:15.406977 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zzck\" (UniqueName: \"kubernetes.io/projected/b52e2578-874a-4d6e-a07f-650b6196ff5b-kube-api-access-7zzck\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m8mqc\" (UID: \"b52e2578-874a-4d6e-a07f-650b6196ff5b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:15.412821 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:15.412790 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc"] Jul 13 19:30:15.508214 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:15.508178 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b52e2578-874a-4d6e-a07f-650b6196ff5b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m8mqc\" (UID: \"b52e2578-874a-4d6e-a07f-650b6196ff5b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:15.508214 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:15.508222 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zzck\" (UniqueName: \"kubernetes.io/projected/b52e2578-874a-4d6e-a07f-650b6196ff5b-kube-api-access-7zzck\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m8mqc\" (UID: \"b52e2578-874a-4d6e-a07f-650b6196ff5b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:15.508592 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:15.508563 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b52e2578-874a-4d6e-a07f-650b6196ff5b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m8mqc\" (UID: \"b52e2578-874a-4d6e-a07f-650b6196ff5b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:15.519032 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:15.519000 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zzck\" (UniqueName: \"kubernetes.io/projected/b52e2578-874a-4d6e-a07f-650b6196ff5b-kube-api-access-7zzck\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m8mqc\" (UID: \"b52e2578-874a-4d6e-a07f-650b6196ff5b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:15.707693 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:15.707599 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:15.854850 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:15.854826 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc"] Jul 13 19:30:15.857257 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:30:15.857227 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb52e2578_874a_4d6e_a07f_650b6196ff5b.slice/crio-2103729959f9ff71726b42a7b35072d6e7049b028d5e46ae85431346581f692e WatchSource:0}: Error finding container 2103729959f9ff71726b42a7b35072d6e7049b028d5e46ae85431346581f692e: Status 404 returned error can't find the container with id 2103729959f9ff71726b42a7b35072d6e7049b028d5e46ae85431346581f692e Jul 13 19:30:16.673713 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:16.673675 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" event={"ID":"b52e2578-874a-4d6e-a07f-650b6196ff5b","Type":"ContainerStarted","Data":"2103729959f9ff71726b42a7b35072d6e7049b028d5e46ae85431346581f692e"} Jul 13 19:30:22.631369 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:22.631299 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-786b65d94-fqdbs" podUID="193869b9-d860-4865-b09f-09abd9d91e15" containerName="console" containerID="cri-o://a9dfdf6179960f1d2f71cdad1de7e9157257813b064e3250bced3595d63fe529" gracePeriod=15 Jul 13 19:30:22.652620 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:22.652595 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x9bzx" Jul 13 19:30:22.895088 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:22.895008 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:30:22.975462 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:22.975424 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-trusted-ca-bundle\") pod \"193869b9-d860-4865-b09f-09abd9d91e15\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " Jul 13 19:30:22.975462 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:22.975472 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5j6w\" (UniqueName: \"kubernetes.io/projected/193869b9-d860-4865-b09f-09abd9d91e15-kube-api-access-k5j6w\") pod \"193869b9-d860-4865-b09f-09abd9d91e15\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " Jul 13 19:30:22.975760 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:22.975519 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-service-ca\") pod \"193869b9-d860-4865-b09f-09abd9d91e15\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " Jul 13 19:30:22.975760 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:22.975537 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-oauth-serving-cert\") pod \"193869b9-d860-4865-b09f-09abd9d91e15\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " Jul 13 19:30:22.975760 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:22.975619 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-console-config\") pod \"193869b9-d860-4865-b09f-09abd9d91e15\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " Jul 13 19:30:22.975760 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:22.975646 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/193869b9-d860-4865-b09f-09abd9d91e15-console-oauth-config\") pod \"193869b9-d860-4865-b09f-09abd9d91e15\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " Jul 13 19:30:22.975760 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:22.975669 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/193869b9-d860-4865-b09f-09abd9d91e15-console-serving-cert\") pod \"193869b9-d860-4865-b09f-09abd9d91e15\" (UID: \"193869b9-d860-4865-b09f-09abd9d91e15\") " Jul 13 19:30:22.976082 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:22.975948 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "193869b9-d860-4865-b09f-09abd9d91e15" (UID: "193869b9-d860-4865-b09f-09abd9d91e15"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:30:22.976165 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:22.976139 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-service-ca" (OuterVolumeSpecName: "service-ca") pod "193869b9-d860-4865-b09f-09abd9d91e15" (UID: "193869b9-d860-4865-b09f-09abd9d91e15"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:30:22.976165 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:22.976147 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-console-config" (OuterVolumeSpecName: "console-config") pod "193869b9-d860-4865-b09f-09abd9d91e15" (UID: "193869b9-d860-4865-b09f-09abd9d91e15"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:30:22.976266 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:22.976212 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "193869b9-d860-4865-b09f-09abd9d91e15" (UID: "193869b9-d860-4865-b09f-09abd9d91e15"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:30:22.977921 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:22.977892 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/193869b9-d860-4865-b09f-09abd9d91e15-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "193869b9-d860-4865-b09f-09abd9d91e15" (UID: "193869b9-d860-4865-b09f-09abd9d91e15"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:30:22.978042 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:22.977964 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/193869b9-d860-4865-b09f-09abd9d91e15-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "193869b9-d860-4865-b09f-09abd9d91e15" (UID: "193869b9-d860-4865-b09f-09abd9d91e15"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:30:22.978042 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:22.977964 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/193869b9-d860-4865-b09f-09abd9d91e15-kube-api-access-k5j6w" (OuterVolumeSpecName: "kube-api-access-k5j6w") pod "193869b9-d860-4865-b09f-09abd9d91e15" (UID: "193869b9-d860-4865-b09f-09abd9d91e15"). InnerVolumeSpecName "kube-api-access-k5j6w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:30:23.076767 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:23.076727 2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-console-config\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:30:23.076767 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:23.076760 2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/193869b9-d860-4865-b09f-09abd9d91e15-console-oauth-config\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:30:23.076767 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:23.076770 2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/193869b9-d860-4865-b09f-09abd9d91e15-console-serving-cert\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:30:23.076767 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:23.076808 2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-trusted-ca-bundle\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:30:23.077084 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:23.076824 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5j6w\" (UniqueName: \"kubernetes.io/projected/193869b9-d860-4865-b09f-09abd9d91e15-kube-api-access-k5j6w\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:30:23.077084 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:23.076836 2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-service-ca\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:30:23.077084 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:23.076848 2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/193869b9-d860-4865-b09f-09abd9d91e15-oauth-serving-cert\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:30:23.718446 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:23.718404 2556 generic.go:356] "Generic (PLEG): container finished" podID="193869b9-d860-4865-b09f-09abd9d91e15" containerID="a9dfdf6179960f1d2f71cdad1de7e9157257813b064e3250bced3595d63fe529" exitCode=0 Jul 13 19:30:23.718995 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:23.718477 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-786b65d94-fqdbs" Jul 13 19:30:23.718995 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:23.718505 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-786b65d94-fqdbs" event={"ID":"193869b9-d860-4865-b09f-09abd9d91e15","Type":"ContainerDied","Data":"a9dfdf6179960f1d2f71cdad1de7e9157257813b064e3250bced3595d63fe529"} Jul 13 19:30:23.718995 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:23.718566 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-786b65d94-fqdbs" event={"ID":"193869b9-d860-4865-b09f-09abd9d91e15","Type":"ContainerDied","Data":"b0dce080ca7bfa5ec01d15646d77fc6a4248e04bab7533520da1000b2848d329"} Jul 13 19:30:23.718995 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:23.718591 2556 scope.go:117] "RemoveContainer" containerID="a9dfdf6179960f1d2f71cdad1de7e9157257813b064e3250bced3595d63fe529" Jul 13 19:30:23.741949 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:23.741910 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-786b65d94-fqdbs"] Jul 13 19:30:23.745456 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:23.745428 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-786b65d94-fqdbs"] Jul 13 19:30:24.260240 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:24.260217 2556 scope.go:117] "RemoveContainer" containerID="a9dfdf6179960f1d2f71cdad1de7e9157257813b064e3250bced3595d63fe529" Jul 13 19:30:24.260623 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:24.260594 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9dfdf6179960f1d2f71cdad1de7e9157257813b064e3250bced3595d63fe529\": container with ID starting with a9dfdf6179960f1d2f71cdad1de7e9157257813b064e3250bced3595d63fe529 not found: ID does not exist" containerID="a9dfdf6179960f1d2f71cdad1de7e9157257813b064e3250bced3595d63fe529" Jul 13 19:30:24.260724 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:24.260631 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9dfdf6179960f1d2f71cdad1de7e9157257813b064e3250bced3595d63fe529"} err="failed to get container status \"a9dfdf6179960f1d2f71cdad1de7e9157257813b064e3250bced3595d63fe529\": rpc error: code = NotFound desc = could not find container \"a9dfdf6179960f1d2f71cdad1de7e9157257813b064e3250bced3595d63fe529\": container with ID starting with a9dfdf6179960f1d2f71cdad1de7e9157257813b064e3250bced3595d63fe529 not found: ID does not exist" Jul 13 19:30:24.727453 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:24.727418 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" event={"ID":"b52e2578-874a-4d6e-a07f-650b6196ff5b","Type":"ContainerStarted","Data":"461cb119b3728e8960bb2df5b5413239d65bbaaf768c9e36bf8cd8a6848c5b3f"} Jul 13 19:30:24.727877 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:24.727527 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:24.748028 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:24.747962 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" podStartSLOduration=1.28955531 podStartE2EDuration="9.747938259s" totalImagesPullingTime="8.458382949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:30:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:30:24.747004377 +0000 UTC m=+519.860912145" watchObservedRunningTime="2026-07-13 19:30:24.747938259 +0000 UTC m=+519.861845969" Jul 13 19:30:24.850939 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:24.850907 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="193869b9-d860-4865-b09f-09abd9d91e15" path="/var/lib/kubelet/pods/193869b9-d860-4865-b09f-09abd9d91e15/volumes" Jul 13 19:30:35.735220 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:35.735181 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:36.776484 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:36.776401 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8"] Jul 13 19:30:36.776882 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:36.776825 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="193869b9-d860-4865-b09f-09abd9d91e15" containerName="console" Jul 13 19:30:36.776882 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:36.776838 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="193869b9-d860-4865-b09f-09abd9d91e15" containerName="console" Jul 13 19:30:36.776965 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:36.776916 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="193869b9-d860-4865-b09f-09abd9d91e15" containerName="console" Jul 13 19:30:36.781451 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:36.781430 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8" Jul 13 19:30:36.806411 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:36.806378 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8"] Jul 13 19:30:36.920535 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:36.920490 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svgqs\" (UniqueName: \"kubernetes.io/projected/e7e2ce20-1cd3-4680-ac73-c74d85bd0538-kube-api-access-svgqs\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rj6q8\" (UID: \"e7e2ce20-1cd3-4680-ac73-c74d85bd0538\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8" Jul 13 19:30:36.920741 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:36.920545 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7e2ce20-1cd3-4680-ac73-c74d85bd0538-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rj6q8\" (UID: \"e7e2ce20-1cd3-4680-ac73-c74d85bd0538\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8" Jul 13 19:30:37.021225 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.021190 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svgqs\" (UniqueName: \"kubernetes.io/projected/e7e2ce20-1cd3-4680-ac73-c74d85bd0538-kube-api-access-svgqs\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rj6q8\" (UID: \"e7e2ce20-1cd3-4680-ac73-c74d85bd0538\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8" Jul 13 19:30:37.021225 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.021232 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7e2ce20-1cd3-4680-ac73-c74d85bd0538-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rj6q8\" (UID: \"e7e2ce20-1cd3-4680-ac73-c74d85bd0538\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8" Jul 13 19:30:37.021655 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.021633 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7e2ce20-1cd3-4680-ac73-c74d85bd0538-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rj6q8\" (UID: \"e7e2ce20-1cd3-4680-ac73-c74d85bd0538\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8" Jul 13 19:30:37.032952 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.032917 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svgqs\" (UniqueName: \"kubernetes.io/projected/e7e2ce20-1cd3-4680-ac73-c74d85bd0538-kube-api-access-svgqs\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rj6q8\" (UID: \"e7e2ce20-1cd3-4680-ac73-c74d85bd0538\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8" Jul 13 19:30:37.092293 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.092248 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8" Jul 13 19:30:37.257569 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.257541 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8"] Jul 13 19:30:37.259355 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:30:37.259330 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7e2ce20_1cd3_4680_ac73_c74d85bd0538.slice/crio-962b489ce5d6a31f8dab799328ee0a1afbb435d028b4f1b7ba1d20ea04cab0cc WatchSource:0}: Error finding container 962b489ce5d6a31f8dab799328ee0a1afbb435d028b4f1b7ba1d20ea04cab0cc: Status 404 returned error can't find the container with id 962b489ce5d6a31f8dab799328ee0a1afbb435d028b4f1b7ba1d20ea04cab0cc Jul 13 19:30:37.558541 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.558206 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8"] Jul 13 19:30:37.562369 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.562326 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8"] Jul 13 19:30:37.567477 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.567437 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc"] Jul 13 19:30:37.567711 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.567685 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" podUID="b52e2578-874a-4d6e-a07f-650b6196ff5b" containerName="manager" containerID="cri-o://461cb119b3728e8960bb2df5b5413239d65bbaaf768c9e36bf8cd8a6848c5b3f" gracePeriod=2 Jul 13 19:30:37.584141 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.584108 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc"] Jul 13 19:30:37.586653 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.586622 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm"] Jul 13 19:30:37.587226 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.587204 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b52e2578-874a-4d6e-a07f-650b6196ff5b" containerName="manager" Jul 13 19:30:37.587653 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.587379 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b52e2578-874a-4d6e-a07f-650b6196ff5b" containerName="manager" Jul 13 19:30:37.587653 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.587403 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7e2ce20-1cd3-4680-ac73-c74d85bd0538" containerName="manager" Jul 13 19:30:37.587653 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.587412 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7e2ce20-1cd3-4680-ac73-c74d85bd0538" containerName="manager" Jul 13 19:30:37.587653 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.587632 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b52e2578-874a-4d6e-a07f-650b6196ff5b" containerName="manager" Jul 13 19:30:37.587653 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.587646 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7e2ce20-1cd3-4680-ac73-c74d85bd0538" containerName="manager" Jul 13 19:30:37.589873 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:37.589830 2556 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"b52e2578-874a-4d6e-a07f-650b6196ff5b\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-m8mqc\": pods \"kuadrant-operator-controller-manager-8684c9ff49-m8mqc\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:37.591875 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.591766 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm" Jul 13 19:30:37.594207 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.594160 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6"] Jul 13 19:30:37.597690 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.597671 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6" Jul 13 19:30:37.603462 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.603415 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm"] Jul 13 19:30:37.613246 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.613205 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6"] Jul 13 19:30:37.625040 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:37.624991 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m8mqc\" is forbidden: User \"system:node:ip-10-0-132-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-20.ec2.internal' and this object" podUID="b52e2578-874a-4d6e-a07f-650b6196ff5b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:37.626631 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:37.626596 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m8mqc\" is forbidden: User \"system:node:ip-10-0-132-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-20.ec2.internal' and this object" podUID="b52e2578-874a-4d6e-a07f-650b6196ff5b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:37.727849 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.727796 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23270c36-da92-4786-87d5-13a09575872d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rc6k6\" (UID: \"23270c36-da92-4786-87d5-13a09575872d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6" Jul 13 19:30:37.728019 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.727877 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d0adfb8-ac08-4882-bd7e-808195b85b2e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qnjgm\" (UID: \"1d0adfb8-ac08-4882-bd7e-808195b85b2e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm" Jul 13 19:30:37.728019 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.727914 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtvsm\" (UniqueName: \"kubernetes.io/projected/1d0adfb8-ac08-4882-bd7e-808195b85b2e-kube-api-access-jtvsm\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qnjgm\" (UID: \"1d0adfb8-ac08-4882-bd7e-808195b85b2e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm" Jul 13 19:30:37.728019 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.727950 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w492x\" (UniqueName: \"kubernetes.io/projected/23270c36-da92-4786-87d5-13a09575872d-kube-api-access-w492x\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rc6k6\" (UID: \"23270c36-da92-4786-87d5-13a09575872d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6" Jul 13 19:30:37.804240 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.804212 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:37.806217 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:37.806177 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m8mqc\" is forbidden: User \"system:node:ip-10-0-132-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-20.ec2.internal' and this object" podUID="b52e2578-874a-4d6e-a07f-650b6196ff5b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:37.806810 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.806768 2556 generic.go:356] "Generic (PLEG): container finished" podID="b52e2578-874a-4d6e-a07f-650b6196ff5b" containerID="461cb119b3728e8960bb2df5b5413239d65bbaaf768c9e36bf8cd8a6848c5b3f" exitCode=0 Jul 13 19:30:37.806906 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.806835 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:37.806906 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.806864 2556 scope.go:117] "RemoveContainer" containerID="461cb119b3728e8960bb2df5b5413239d65bbaaf768c9e36bf8cd8a6848c5b3f" Jul 13 19:30:37.808654 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:37.808586 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m8mqc\" is forbidden: User \"system:node:ip-10-0-132-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-20.ec2.internal' and this object" podUID="b52e2578-874a-4d6e-a07f-650b6196ff5b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:37.808962 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:37.808943 2556 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8" container="manager" Jul 13 19:30:37.810504 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:37.810483 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m8mqc\" is forbidden: User \"system:node:ip-10-0-132-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-20.ec2.internal' and this object" podUID="b52e2578-874a-4d6e-a07f-650b6196ff5b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:37.812039 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:37.812011 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-rj6q8\" is forbidden: User \"system:node:ip-10-0-132-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-20.ec2.internal' and this object" podUID="e7e2ce20-1cd3-4680-ac73-c74d85bd0538" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8" Jul 13 19:30:37.818174 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.818138 2556 scope.go:117] "RemoveContainer" containerID="461cb119b3728e8960bb2df5b5413239d65bbaaf768c9e36bf8cd8a6848c5b3f" Jul 13 19:30:37.818556 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:37.818530 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"461cb119b3728e8960bb2df5b5413239d65bbaaf768c9e36bf8cd8a6848c5b3f\": container with ID starting with 461cb119b3728e8960bb2df5b5413239d65bbaaf768c9e36bf8cd8a6848c5b3f not found: ID does not exist" containerID="461cb119b3728e8960bb2df5b5413239d65bbaaf768c9e36bf8cd8a6848c5b3f" Jul 13 19:30:37.818659 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.818572 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"461cb119b3728e8960bb2df5b5413239d65bbaaf768c9e36bf8cd8a6848c5b3f"} err="failed to get container status \"461cb119b3728e8960bb2df5b5413239d65bbaaf768c9e36bf8cd8a6848c5b3f\": rpc error: code = NotFound desc = could not find container \"461cb119b3728e8960bb2df5b5413239d65bbaaf768c9e36bf8cd8a6848c5b3f\": container with ID starting with 461cb119b3728e8960bb2df5b5413239d65bbaaf768c9e36bf8cd8a6848c5b3f not found: ID does not exist" Jul 13 19:30:37.829380 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.829340 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtvsm\" (UniqueName: \"kubernetes.io/projected/1d0adfb8-ac08-4882-bd7e-808195b85b2e-kube-api-access-jtvsm\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qnjgm\" (UID: \"1d0adfb8-ac08-4882-bd7e-808195b85b2e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm" Jul 13 19:30:37.829589 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.829392 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w492x\" (UniqueName: \"kubernetes.io/projected/23270c36-da92-4786-87d5-13a09575872d-kube-api-access-w492x\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rc6k6\" (UID: \"23270c36-da92-4786-87d5-13a09575872d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6" Jul 13 19:30:37.829589 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.829474 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23270c36-da92-4786-87d5-13a09575872d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rc6k6\" (UID: \"23270c36-da92-4786-87d5-13a09575872d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6" Jul 13 19:30:37.829589 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.829527 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d0adfb8-ac08-4882-bd7e-808195b85b2e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qnjgm\" (UID: \"1d0adfb8-ac08-4882-bd7e-808195b85b2e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm" Jul 13 19:30:37.829911 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.829893 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23270c36-da92-4786-87d5-13a09575872d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rc6k6\" (UID: \"23270c36-da92-4786-87d5-13a09575872d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6" Jul 13 19:30:37.829970 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.829890 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d0adfb8-ac08-4882-bd7e-808195b85b2e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qnjgm\" (UID: \"1d0adfb8-ac08-4882-bd7e-808195b85b2e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm" Jul 13 19:30:37.839738 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.839706 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtvsm\" (UniqueName: \"kubernetes.io/projected/1d0adfb8-ac08-4882-bd7e-808195b85b2e-kube-api-access-jtvsm\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qnjgm\" (UID: \"1d0adfb8-ac08-4882-bd7e-808195b85b2e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm" Jul 13 19:30:37.839929 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.839795 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w492x\" (UniqueName: \"kubernetes.io/projected/23270c36-da92-4786-87d5-13a09575872d-kube-api-access-w492x\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rc6k6\" (UID: \"23270c36-da92-4786-87d5-13a09575872d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6" Jul 13 19:30:37.930234 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.930190 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zzck\" (UniqueName: \"kubernetes.io/projected/b52e2578-874a-4d6e-a07f-650b6196ff5b-kube-api-access-7zzck\") pod \"b52e2578-874a-4d6e-a07f-650b6196ff5b\" (UID: \"b52e2578-874a-4d6e-a07f-650b6196ff5b\") " Jul 13 19:30:37.930424 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.930262 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b52e2578-874a-4d6e-a07f-650b6196ff5b-extensions-socket-volume\") pod \"b52e2578-874a-4d6e-a07f-650b6196ff5b\" (UID: \"b52e2578-874a-4d6e-a07f-650b6196ff5b\") " Jul 13 19:30:37.930737 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.930714 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b52e2578-874a-4d6e-a07f-650b6196ff5b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b52e2578-874a-4d6e-a07f-650b6196ff5b" (UID: "b52e2578-874a-4d6e-a07f-650b6196ff5b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:30:37.932397 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.932374 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b52e2578-874a-4d6e-a07f-650b6196ff5b-kube-api-access-7zzck" (OuterVolumeSpecName: "kube-api-access-7zzck") pod "b52e2578-874a-4d6e-a07f-650b6196ff5b" (UID: "b52e2578-874a-4d6e-a07f-650b6196ff5b"). InnerVolumeSpecName "kube-api-access-7zzck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:30:37.946651 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.946615 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm" Jul 13 19:30:37.954522 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:37.954495 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6" Jul 13 19:30:38.031929 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:38.031895 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zzck\" (UniqueName: \"kubernetes.io/projected/b52e2578-874a-4d6e-a07f-650b6196ff5b-kube-api-access-7zzck\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:30:38.031929 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:38.031926 2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b52e2578-874a-4d6e-a07f-650b6196ff5b-extensions-socket-volume\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:30:38.118668 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:38.118640 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm"] Jul 13 19:30:38.120616 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:38.120582 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m8mqc\" is forbidden: User \"system:node:ip-10-0-132-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-20.ec2.internal' and this object" podUID="b52e2578-874a-4d6e-a07f-650b6196ff5b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:38.122041 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:38.122008 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-rj6q8\" is forbidden: User \"system:node:ip-10-0-132-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-20.ec2.internal' and this object" podUID="e7e2ce20-1cd3-4680-ac73-c74d85bd0538" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8" Jul 13 19:30:38.122641 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:30:38.122619 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d0adfb8_ac08_4882_bd7e_808195b85b2e.slice/crio-b72af927a598cbfdc41628784e293be8410e1688002aba6bf09d7e949bd328d6 WatchSource:0}: Error finding container b72af927a598cbfdc41628784e293be8410e1688002aba6bf09d7e949bd328d6: Status 404 returned error can't find the container with id b72af927a598cbfdc41628784e293be8410e1688002aba6bf09d7e949bd328d6 Jul 13 19:30:38.136457 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:38.136434 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6"] Jul 13 19:30:38.138412 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:30:38.138385 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23270c36_da92_4786_87d5_13a09575872d.slice/crio-0b3ecafd913101e3075968146a99165085e4ca106db6362bb52a1a1133efaba9 WatchSource:0}: Error finding container 0b3ecafd913101e3075968146a99165085e4ca106db6362bb52a1a1133efaba9: Status 404 returned error can't find the container with id 0b3ecafd913101e3075968146a99165085e4ca106db6362bb52a1a1133efaba9 Jul 13 19:30:38.818426 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:38.818385 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm" event={"ID":"1d0adfb8-ac08-4882-bd7e-808195b85b2e","Type":"ContainerStarted","Data":"35a8557f3faf8574f7efcdc839c9393127bd6ee5031f0a950ccd59c6733c82a7"} Jul 13 19:30:38.818882 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:38.818461 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm" Jul 13 19:30:38.818882 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:38.818493 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm" event={"ID":"1d0adfb8-ac08-4882-bd7e-808195b85b2e","Type":"ContainerStarted","Data":"b72af927a598cbfdc41628784e293be8410e1688002aba6bf09d7e949bd328d6"} Jul 13 19:30:38.820060 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:38.820021 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m8mqc\" is forbidden: User \"system:node:ip-10-0-132-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-20.ec2.internal' and this object" podUID="b52e2578-874a-4d6e-a07f-650b6196ff5b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:38.820296 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:38.820274 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8" podUID="e7e2ce20-1cd3-4680-ac73-c74d85bd0538" containerName="manager" containerID="cri-o://8ced9fbf84d905185e49390a8ba2573422ee261edbc83a3c20e35f01a42a3a3c" gracePeriod=2 Jul 13 19:30:38.821361 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:38.821317 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-rj6q8\" is forbidden: User \"system:node:ip-10-0-132-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-20.ec2.internal' and this object" podUID="e7e2ce20-1cd3-4680-ac73-c74d85bd0538" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8" Jul 13 19:30:38.822273 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:38.822247 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6" event={"ID":"23270c36-da92-4786-87d5-13a09575872d","Type":"ContainerStarted","Data":"a742d9d0e7fbbe71aa76cd196d1d120160107b5e14e9b144f4481f1fe5d9b5d2"} Jul 13 19:30:38.822364 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:38.822296 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6" event={"ID":"23270c36-da92-4786-87d5-13a09575872d","Type":"ContainerStarted","Data":"0b3ecafd913101e3075968146a99165085e4ca106db6362bb52a1a1133efaba9"} Jul 13 19:30:38.822364 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:38.822317 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6" Jul 13 19:30:38.846714 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:38.846655 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm" podStartSLOduration=1.846635451 podStartE2EDuration="1.846635451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:30:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:30:38.845106153 +0000 UTC m=+533.959013862" watchObservedRunningTime="2026-07-13 19:30:38.846635451 +0000 UTC m=+533.960543163" Jul 13 19:30:38.847059 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:38.847007 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m8mqc\" is forbidden: User \"system:node:ip-10-0-132-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-20.ec2.internal' and this object" podUID="b52e2578-874a-4d6e-a07f-650b6196ff5b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m8mqc" Jul 13 19:30:38.848447 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:38.848414 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-rj6q8\" is forbidden: User \"system:node:ip-10-0-132-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-20.ec2.internal' and this object" podUID="e7e2ce20-1cd3-4680-ac73-c74d85bd0538" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8" Jul 13 19:30:38.853003 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:38.852980 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b52e2578-874a-4d6e-a07f-650b6196ff5b" path="/var/lib/kubelet/pods/b52e2578-874a-4d6e-a07f-650b6196ff5b/volumes" Jul 13 19:30:38.867584 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:38.867531 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6" podStartSLOduration=1.867513678 podStartE2EDuration="1.867513678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:30:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:30:38.865094113 +0000 UTC m=+533.979001822" watchObservedRunningTime="2026-07-13 19:30:38.867513678 +0000 UTC m=+533.981421387" Jul 13 19:30:39.083415 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:39.083346 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8" Jul 13 19:30:39.243826 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:39.243765 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svgqs\" (UniqueName: \"kubernetes.io/projected/e7e2ce20-1cd3-4680-ac73-c74d85bd0538-kube-api-access-svgqs\") pod \"e7e2ce20-1cd3-4680-ac73-c74d85bd0538\" (UID: \"e7e2ce20-1cd3-4680-ac73-c74d85bd0538\") " Jul 13 19:30:39.244012 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:39.243943 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7e2ce20-1cd3-4680-ac73-c74d85bd0538-extensions-socket-volume\") pod \"e7e2ce20-1cd3-4680-ac73-c74d85bd0538\" (UID: \"e7e2ce20-1cd3-4680-ac73-c74d85bd0538\") " Jul 13 19:30:39.244378 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:39.244331 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7e2ce20-1cd3-4680-ac73-c74d85bd0538-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e7e2ce20-1cd3-4680-ac73-c74d85bd0538" (UID: "e7e2ce20-1cd3-4680-ac73-c74d85bd0538"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:30:39.246529 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:39.246498 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7e2ce20-1cd3-4680-ac73-c74d85bd0538-kube-api-access-svgqs" (OuterVolumeSpecName: "kube-api-access-svgqs") pod "e7e2ce20-1cd3-4680-ac73-c74d85bd0538" (UID: "e7e2ce20-1cd3-4680-ac73-c74d85bd0538"). InnerVolumeSpecName "kube-api-access-svgqs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:30:39.344922 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:39.344830 2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7e2ce20-1cd3-4680-ac73-c74d85bd0538-extensions-socket-volume\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:30:39.344922 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:39.344869 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svgqs\" (UniqueName: \"kubernetes.io/projected/e7e2ce20-1cd3-4680-ac73-c74d85bd0538-kube-api-access-svgqs\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:30:39.829849 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:39.829811 2556 generic.go:356] "Generic (PLEG): container finished" podID="e7e2ce20-1cd3-4680-ac73-c74d85bd0538" containerID="8ced9fbf84d905185e49390a8ba2573422ee261edbc83a3c20e35f01a42a3a3c" exitCode=0 Jul 13 19:30:39.829849 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:39.829847 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8" Jul 13 19:30:39.830407 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:39.829907 2556 scope.go:117] "RemoveContainer" containerID="8ced9fbf84d905185e49390a8ba2573422ee261edbc83a3c20e35f01a42a3a3c" Jul 13 19:30:39.831964 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:39.831931 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-rj6q8\" is forbidden: User \"system:node:ip-10-0-132-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-20.ec2.internal' and this object" podUID="e7e2ce20-1cd3-4680-ac73-c74d85bd0538" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8" Jul 13 19:30:39.842419 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:39.842378 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-rj6q8\" is forbidden: User \"system:node:ip-10-0-132-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-20.ec2.internal' and this object" podUID="e7e2ce20-1cd3-4680-ac73-c74d85bd0538" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rj6q8" Jul 13 19:30:39.843893 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:39.843864 2556 scope.go:117] "RemoveContainer" containerID="8ced9fbf84d905185e49390a8ba2573422ee261edbc83a3c20e35f01a42a3a3c" Jul 13 19:30:39.844204 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:39.844183 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ced9fbf84d905185e49390a8ba2573422ee261edbc83a3c20e35f01a42a3a3c\": container with ID starting with 8ced9fbf84d905185e49390a8ba2573422ee261edbc83a3c20e35f01a42a3a3c not found: ID does not exist" containerID="8ced9fbf84d905185e49390a8ba2573422ee261edbc83a3c20e35f01a42a3a3c" Jul 13 19:30:39.844256 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:39.844214 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ced9fbf84d905185e49390a8ba2573422ee261edbc83a3c20e35f01a42a3a3c"} err="failed to get container status \"8ced9fbf84d905185e49390a8ba2573422ee261edbc83a3c20e35f01a42a3a3c\": rpc error: code = NotFound desc = could not find container \"8ced9fbf84d905185e49390a8ba2573422ee261edbc83a3c20e35f01a42a3a3c\": container with ID starting with 8ced9fbf84d905185e49390a8ba2573422ee261edbc83a3c20e35f01a42a3a3c not found: ID does not exist" Jul 13 19:30:40.849879 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:40.849844 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7e2ce20-1cd3-4680-ac73-c74d85bd0538" path="/var/lib/kubelet/pods/e7e2ce20-1cd3-4680-ac73-c74d85bd0538/volumes" Jul 13 19:30:49.832601 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:49.832567 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm" Jul 13 19:30:49.833031 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:49.832735 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6" Jul 13 19:30:49.948720 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:49.948685 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6"] Jul 13 19:30:49.948958 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:49.948907 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6" podUID="23270c36-da92-4786-87d5-13a09575872d" containerName="manager" containerID="cri-o://a742d9d0e7fbbe71aa76cd196d1d120160107b5e14e9b144f4481f1fe5d9b5d2" gracePeriod=10 Jul 13 19:30:50.213655 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.213628 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6" Jul 13 19:30:50.310296 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.310257 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6"] Jul 13 19:30:50.310704 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.310692 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23270c36-da92-4786-87d5-13a09575872d" containerName="manager" Jul 13 19:30:50.310751 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.310705 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="23270c36-da92-4786-87d5-13a09575872d" containerName="manager" Jul 13 19:30:50.310815 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.310804 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="23270c36-da92-4786-87d5-13a09575872d" containerName="manager" Jul 13 19:30:50.312983 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.312967 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6" Jul 13 19:30:50.327116 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.327086 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6"] Jul 13 19:30:50.357164 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.357122 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w492x\" (UniqueName: \"kubernetes.io/projected/23270c36-da92-4786-87d5-13a09575872d-kube-api-access-w492x\") pod \"23270c36-da92-4786-87d5-13a09575872d\" (UID: \"23270c36-da92-4786-87d5-13a09575872d\") " Jul 13 19:30:50.357385 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.357217 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23270c36-da92-4786-87d5-13a09575872d-extensions-socket-volume\") pod \"23270c36-da92-4786-87d5-13a09575872d\" (UID: \"23270c36-da92-4786-87d5-13a09575872d\") " Jul 13 19:30:50.357638 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.357612 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23270c36-da92-4786-87d5-13a09575872d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "23270c36-da92-4786-87d5-13a09575872d" (UID: "23270c36-da92-4786-87d5-13a09575872d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:30:50.359402 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.359370 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23270c36-da92-4786-87d5-13a09575872d-kube-api-access-w492x" (OuterVolumeSpecName: "kube-api-access-w492x") pod "23270c36-da92-4786-87d5-13a09575872d" (UID: "23270c36-da92-4786-87d5-13a09575872d"). InnerVolumeSpecName "kube-api-access-w492x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:30:50.458301 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.458198 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrsld\" (UniqueName: \"kubernetes.io/projected/1098758b-2aaa-4672-a073-4dd17ab700b6-kube-api-access-mrsld\") pod \"kuadrant-operator-controller-manager-86878c769f-nkzg6\" (UID: \"1098758b-2aaa-4672-a073-4dd17ab700b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6" Jul 13 19:30:50.458301 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.458238 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1098758b-2aaa-4672-a073-4dd17ab700b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nkzg6\" (UID: \"1098758b-2aaa-4672-a073-4dd17ab700b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6" Jul 13 19:30:50.458489 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.458411 2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23270c36-da92-4786-87d5-13a09575872d-extensions-socket-volume\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:30:50.458489 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.458435 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w492x\" (UniqueName: \"kubernetes.io/projected/23270c36-da92-4786-87d5-13a09575872d-kube-api-access-w492x\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:30:50.559501 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.559452 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrsld\" (UniqueName: \"kubernetes.io/projected/1098758b-2aaa-4672-a073-4dd17ab700b6-kube-api-access-mrsld\") pod \"kuadrant-operator-controller-manager-86878c769f-nkzg6\" (UID: \"1098758b-2aaa-4672-a073-4dd17ab700b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6" Jul 13 19:30:50.559501 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.559502 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1098758b-2aaa-4672-a073-4dd17ab700b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nkzg6\" (UID: \"1098758b-2aaa-4672-a073-4dd17ab700b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6" Jul 13 19:30:50.559967 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.559945 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1098758b-2aaa-4672-a073-4dd17ab700b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nkzg6\" (UID: \"1098758b-2aaa-4672-a073-4dd17ab700b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6" Jul 13 19:30:50.570063 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.570027 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrsld\" (UniqueName: \"kubernetes.io/projected/1098758b-2aaa-4672-a073-4dd17ab700b6-kube-api-access-mrsld\") pod \"kuadrant-operator-controller-manager-86878c769f-nkzg6\" (UID: \"1098758b-2aaa-4672-a073-4dd17ab700b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6" Jul 13 19:30:50.623977 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.623935 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6" Jul 13 19:30:50.771676 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.771653 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6"] Jul 13 19:30:50.774236 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:30:50.774203 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1098758b_2aaa_4672_a073_4dd17ab700b6.slice/crio-4e03156adf42e2cfa9461d1d5adb4cf20998d17986156cdf8e6dca87d455c77c WatchSource:0}: Error finding container 4e03156adf42e2cfa9461d1d5adb4cf20998d17986156cdf8e6dca87d455c77c: Status 404 returned error can't find the container with id 4e03156adf42e2cfa9461d1d5adb4cf20998d17986156cdf8e6dca87d455c77c Jul 13 19:30:50.897648 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.897609 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6" event={"ID":"1098758b-2aaa-4672-a073-4dd17ab700b6","Type":"ContainerStarted","Data":"482a74c42c9a97971116b670088a152778b4b79967496bc36465378131be842e"} Jul 13 19:30:50.898182 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.897678 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6" Jul 13 19:30:50.898182 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.897702 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6" event={"ID":"1098758b-2aaa-4672-a073-4dd17ab700b6","Type":"ContainerStarted","Data":"4e03156adf42e2cfa9461d1d5adb4cf20998d17986156cdf8e6dca87d455c77c"} Jul 13 19:30:50.899091 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.899065 2556 generic.go:356] "Generic (PLEG): container finished" podID="23270c36-da92-4786-87d5-13a09575872d" containerID="a742d9d0e7fbbe71aa76cd196d1d120160107b5e14e9b144f4481f1fe5d9b5d2" exitCode=0 Jul 13 19:30:50.899215 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.899119 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6" Jul 13 19:30:50.899215 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.899142 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6" event={"ID":"23270c36-da92-4786-87d5-13a09575872d","Type":"ContainerDied","Data":"a742d9d0e7fbbe71aa76cd196d1d120160107b5e14e9b144f4481f1fe5d9b5d2"} Jul 13 19:30:50.899215 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.899189 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6" event={"ID":"23270c36-da92-4786-87d5-13a09575872d","Type":"ContainerDied","Data":"0b3ecafd913101e3075968146a99165085e4ca106db6362bb52a1a1133efaba9"} Jul 13 19:30:50.899215 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.899213 2556 scope.go:117] "RemoveContainer" containerID="a742d9d0e7fbbe71aa76cd196d1d120160107b5e14e9b144f4481f1fe5d9b5d2" Jul 13 19:30:50.909406 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.909386 2556 scope.go:117] "RemoveContainer" containerID="a742d9d0e7fbbe71aa76cd196d1d120160107b5e14e9b144f4481f1fe5d9b5d2" Jul 13 19:30:50.909731 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:30:50.909712 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a742d9d0e7fbbe71aa76cd196d1d120160107b5e14e9b144f4481f1fe5d9b5d2\": container with ID starting with a742d9d0e7fbbe71aa76cd196d1d120160107b5e14e9b144f4481f1fe5d9b5d2 not found: ID does not exist" containerID="a742d9d0e7fbbe71aa76cd196d1d120160107b5e14e9b144f4481f1fe5d9b5d2" Jul 13 19:30:50.909812 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.909742 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a742d9d0e7fbbe71aa76cd196d1d120160107b5e14e9b144f4481f1fe5d9b5d2"} err="failed to get container status \"a742d9d0e7fbbe71aa76cd196d1d120160107b5e14e9b144f4481f1fe5d9b5d2\": rpc error: code = NotFound desc = could not find container \"a742d9d0e7fbbe71aa76cd196d1d120160107b5e14e9b144f4481f1fe5d9b5d2\": container with ID starting with a742d9d0e7fbbe71aa76cd196d1d120160107b5e14e9b144f4481f1fe5d9b5d2 not found: ID does not exist" Jul 13 19:30:50.916505 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.916452 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6" podStartSLOduration=0.916435122 podStartE2EDuration="916.435122ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:30:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:30:50.915469884 +0000 UTC m=+546.029377586" watchObservedRunningTime="2026-07-13 19:30:50.916435122 +0000 UTC m=+546.030342830" Jul 13 19:30:50.932534 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.932498 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6"] Jul 13 19:30:50.938849 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:50.938814 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rc6k6"] Jul 13 19:30:52.849887 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:30:52.849853 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23270c36-da92-4786-87d5-13a09575872d" path="/var/lib/kubelet/pods/23270c36-da92-4786-87d5-13a09575872d/volumes" Jul 13 19:31:01.908391 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:01.908354 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6" Jul 13 19:31:01.985271 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:01.985237 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm"] Jul 13 19:31:01.985499 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:01.985458 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm" podUID="1d0adfb8-ac08-4882-bd7e-808195b85b2e" containerName="manager" containerID="cri-o://35a8557f3faf8574f7efcdc839c9393127bd6ee5031f0a950ccd59c6733c82a7" gracePeriod=10 Jul 13 19:31:02.253695 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:02.253662 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm" Jul 13 19:31:02.379632 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:02.379594 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d0adfb8-ac08-4882-bd7e-808195b85b2e-extensions-socket-volume\") pod \"1d0adfb8-ac08-4882-bd7e-808195b85b2e\" (UID: \"1d0adfb8-ac08-4882-bd7e-808195b85b2e\") " Jul 13 19:31:02.379873 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:02.379656 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtvsm\" (UniqueName: \"kubernetes.io/projected/1d0adfb8-ac08-4882-bd7e-808195b85b2e-kube-api-access-jtvsm\") pod \"1d0adfb8-ac08-4882-bd7e-808195b85b2e\" (UID: \"1d0adfb8-ac08-4882-bd7e-808195b85b2e\") " Jul 13 19:31:02.380149 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:02.380119 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d0adfb8-ac08-4882-bd7e-808195b85b2e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1d0adfb8-ac08-4882-bd7e-808195b85b2e" (UID: "1d0adfb8-ac08-4882-bd7e-808195b85b2e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:31:02.381900 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:02.381876 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d0adfb8-ac08-4882-bd7e-808195b85b2e-kube-api-access-jtvsm" (OuterVolumeSpecName: "kube-api-access-jtvsm") pod "1d0adfb8-ac08-4882-bd7e-808195b85b2e" (UID: "1d0adfb8-ac08-4882-bd7e-808195b85b2e"). InnerVolumeSpecName "kube-api-access-jtvsm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:31:02.480723 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:02.480674 2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d0adfb8-ac08-4882-bd7e-808195b85b2e-extensions-socket-volume\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:31:02.480723 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:02.480713 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtvsm\" (UniqueName: \"kubernetes.io/projected/1d0adfb8-ac08-4882-bd7e-808195b85b2e-kube-api-access-jtvsm\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:31:02.969944 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:02.969904 2556 generic.go:356] "Generic (PLEG): container finished" podID="1d0adfb8-ac08-4882-bd7e-808195b85b2e" containerID="35a8557f3faf8574f7efcdc839c9393127bd6ee5031f0a950ccd59c6733c82a7" exitCode=0 Jul 13 19:31:02.970451 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:02.969961 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm" Jul 13 19:31:02.970451 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:02.969994 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm" event={"ID":"1d0adfb8-ac08-4882-bd7e-808195b85b2e","Type":"ContainerDied","Data":"35a8557f3faf8574f7efcdc839c9393127bd6ee5031f0a950ccd59c6733c82a7"} Jul 13 19:31:02.970451 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:02.970050 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm" event={"ID":"1d0adfb8-ac08-4882-bd7e-808195b85b2e","Type":"ContainerDied","Data":"b72af927a598cbfdc41628784e293be8410e1688002aba6bf09d7e949bd328d6"} Jul 13 19:31:02.970451 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:02.970073 2556 scope.go:117] "RemoveContainer" containerID="35a8557f3faf8574f7efcdc839c9393127bd6ee5031f0a950ccd59c6733c82a7" Jul 13 19:31:02.979349 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:02.979331 2556 scope.go:117] "RemoveContainer" containerID="35a8557f3faf8574f7efcdc839c9393127bd6ee5031f0a950ccd59c6733c82a7" Jul 13 19:31:02.979710 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:31:02.979682 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35a8557f3faf8574f7efcdc839c9393127bd6ee5031f0a950ccd59c6733c82a7\": container with ID starting with 35a8557f3faf8574f7efcdc839c9393127bd6ee5031f0a950ccd59c6733c82a7 not found: ID does not exist" containerID="35a8557f3faf8574f7efcdc839c9393127bd6ee5031f0a950ccd59c6733c82a7" Jul 13 19:31:02.979756 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:02.979722 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a8557f3faf8574f7efcdc839c9393127bd6ee5031f0a950ccd59c6733c82a7"} err="failed to get container status \"35a8557f3faf8574f7efcdc839c9393127bd6ee5031f0a950ccd59c6733c82a7\": rpc error: code = NotFound desc = could not find container \"35a8557f3faf8574f7efcdc839c9393127bd6ee5031f0a950ccd59c6733c82a7\": container with ID starting with 35a8557f3faf8574f7efcdc839c9393127bd6ee5031f0a950ccd59c6733c82a7 not found: ID does not exist" Jul 13 19:31:03.011127 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:03.011085 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm"] Jul 13 19:31:03.013577 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:03.013549 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qnjgm"] Jul 13 19:31:04.849364 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:04.849332 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d0adfb8-ac08-4882-bd7e-808195b85b2e" path="/var/lib/kubelet/pods/1d0adfb8-ac08-4882-bd7e-808195b85b2e/volumes" Jul 13 19:31:06.412793 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.412747 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm"] Jul 13 19:31:06.413260 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.413245 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d0adfb8-ac08-4882-bd7e-808195b85b2e" containerName="manager" Jul 13 19:31:06.413317 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.413262 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d0adfb8-ac08-4882-bd7e-808195b85b2e" containerName="manager" Jul 13 19:31:06.413362 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.413351 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d0adfb8-ac08-4882-bd7e-808195b85b2e" containerName="manager" Jul 13 19:31:06.416393 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.416366 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.418974 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.418946 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-x8dwp\"" Jul 13 19:31:06.437895 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.437860 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm"] Jul 13 19:31:06.523956 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.523908 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16e74623-f867-449e-a739-4234322bb3e1-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.524160 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.523970 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16e74623-f867-449e-a739-4234322bb3e1-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.524160 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.524026 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br6rc\" (UniqueName: \"kubernetes.io/projected/16e74623-f867-449e-a739-4234322bb3e1-kube-api-access-br6rc\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.524160 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.524105 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16e74623-f867-449e-a739-4234322bb3e1-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.524160 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.524147 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16e74623-f867-449e-a739-4234322bb3e1-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.524348 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.524170 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16e74623-f867-449e-a739-4234322bb3e1-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.524348 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.524205 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16e74623-f867-449e-a739-4234322bb3e1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.524348 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.524234 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16e74623-f867-449e-a739-4234322bb3e1-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.524348 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.524257 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16e74623-f867-449e-a739-4234322bb3e1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.625788 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.625749 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16e74623-f867-449e-a739-4234322bb3e1-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.626001 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.625838 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16e74623-f867-449e-a739-4234322bb3e1-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.626001 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.625867 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16e74623-f867-449e-a739-4234322bb3e1-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.626126 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.626015 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16e74623-f867-449e-a739-4234322bb3e1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.626126 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.626055 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16e74623-f867-449e-a739-4234322bb3e1-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.626126 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.626092 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16e74623-f867-449e-a739-4234322bb3e1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.626279 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.626142 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16e74623-f867-449e-a739-4234322bb3e1-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.626279 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.626199 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16e74623-f867-449e-a739-4234322bb3e1-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.626279 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.626225 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br6rc\" (UniqueName: \"kubernetes.io/projected/16e74623-f867-449e-a739-4234322bb3e1-kube-api-access-br6rc\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.626279 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.626271 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16e74623-f867-449e-a739-4234322bb3e1-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.626491 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.626307 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16e74623-f867-449e-a739-4234322bb3e1-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.626551 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.626525 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16e74623-f867-449e-a739-4234322bb3e1-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.626639 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.626610 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16e74623-f867-449e-a739-4234322bb3e1-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.626861 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.626835 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16e74623-f867-449e-a739-4234322bb3e1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.628598 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.628572 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16e74623-f867-449e-a739-4234322bb3e1-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.628929 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.628907 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16e74623-f867-449e-a739-4234322bb3e1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.641531 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.641492 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16e74623-f867-449e-a739-4234322bb3e1-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.643972 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.643940 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-br6rc\" (UniqueName: \"kubernetes.io/projected/16e74623-f867-449e-a739-4234322bb3e1-kube-api-access-br6rc\") pod \"maas-default-gateway-openshift-default-56654dcf7-l8mnm\" (UID: \"16e74623-f867-449e-a739-4234322bb3e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.728031 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.727920 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:06.876122 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.876093 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm"] Jul 13 19:31:06.879068 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:31:06.879035 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16e74623_f867_449e_a739_4234322bb3e1.slice/crio-445085a62c640d6533252ee72fc62605e6e9d8160620310099f8f207c3b533bf WatchSource:0}: Error finding container 445085a62c640d6533252ee72fc62605e6e9d8160620310099f8f207c3b533bf: Status 404 returned error can't find the container with id 445085a62c640d6533252ee72fc62605e6e9d8160620310099f8f207c3b533bf Jul 13 19:31:06.881253 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.881219 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 19:31:06.881350 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.881289 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 19:31:06.881350 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.881315 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 19:31:06.996712 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.996601 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" event={"ID":"16e74623-f867-449e-a739-4234322bb3e1","Type":"ContainerStarted","Data":"9649ab91bc4a7a6d1d862e46c3fb00159f375e8fed49a954ca0802be94b5778a"} Jul 13 19:31:06.996712 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:06.996668 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" event={"ID":"16e74623-f867-449e-a739-4234322bb3e1","Type":"ContainerStarted","Data":"445085a62c640d6533252ee72fc62605e6e9d8160620310099f8f207c3b533bf"} Jul 13 19:31:07.018620 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:07.018559 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" podStartSLOduration=1.018536678 podStartE2EDuration="1.018536678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:31:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:31:07.016924867 +0000 UTC m=+562.130832576" watchObservedRunningTime="2026-07-13 19:31:07.018536678 +0000 UTC m=+562.132444395" Jul 13 19:31:07.728859 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:07.728814 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:07.733878 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:07.733848 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:08.003040 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:08.002954 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:08.003916 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:08.003891 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:31:24.388823 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:24.388766 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:31:24.391722 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:24.391700 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-tcvzv" Jul 13 19:31:24.393963 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:24.393932 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-88br4\"" Jul 13 19:31:24.394219 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:24.394198 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 13 19:31:24.408229 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:24.408197 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:31:24.456069 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:24.456030 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:31:24.501420 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:24.501370 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b0f25a86-9f3a-4451-b67f-8adada658cd6-config-file\") pod \"limitador-limitador-54d6d988d5-tcvzv\" (UID: \"b0f25a86-9f3a-4451-b67f-8adada658cd6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tcvzv" Jul 13 19:31:24.501420 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:24.501416 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxjsx\" (UniqueName: \"kubernetes.io/projected/b0f25a86-9f3a-4451-b67f-8adada658cd6-kube-api-access-vxjsx\") pod \"limitador-limitador-54d6d988d5-tcvzv\" (UID: \"b0f25a86-9f3a-4451-b67f-8adada658cd6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tcvzv" Jul 13 19:31:24.602495 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:24.602460 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b0f25a86-9f3a-4451-b67f-8adada658cd6-config-file\") pod \"limitador-limitador-54d6d988d5-tcvzv\" (UID: \"b0f25a86-9f3a-4451-b67f-8adada658cd6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tcvzv" Jul 13 19:31:24.602495 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:24.602494 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxjsx\" (UniqueName: \"kubernetes.io/projected/b0f25a86-9f3a-4451-b67f-8adada658cd6-kube-api-access-vxjsx\") pod \"limitador-limitador-54d6d988d5-tcvzv\" (UID: \"b0f25a86-9f3a-4451-b67f-8adada658cd6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tcvzv" Jul 13 19:31:24.603211 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:24.603185 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b0f25a86-9f3a-4451-b67f-8adada658cd6-config-file\") pod \"limitador-limitador-54d6d988d5-tcvzv\" (UID: \"b0f25a86-9f3a-4451-b67f-8adada658cd6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tcvzv" Jul 13 19:31:24.611433 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:24.611401 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxjsx\" (UniqueName: \"kubernetes.io/projected/b0f25a86-9f3a-4451-b67f-8adada658cd6-kube-api-access-vxjsx\") pod \"limitador-limitador-54d6d988d5-tcvzv\" (UID: \"b0f25a86-9f3a-4451-b67f-8adada658cd6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tcvzv" Jul 13 19:31:24.703614 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:24.703513 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-tcvzv" Jul 13 19:31:24.859390 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:24.859361 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:31:24.861056 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:31:24.861029 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f25a86_9f3a_4451_b67f_8adada658cd6.slice/crio-29ca3ee7a0653dad5bf2c392a247f8f7e70bc8cdde8145ea995f97f4782887d8 WatchSource:0}: Error finding container 29ca3ee7a0653dad5bf2c392a247f8f7e70bc8cdde8145ea995f97f4782887d8: Status 404 returned error can't find the container with id 29ca3ee7a0653dad5bf2c392a247f8f7e70bc8cdde8145ea995f97f4782887d8 Jul 13 19:31:25.103592 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:25.103552 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-tcvzv" event={"ID":"b0f25a86-9f3a-4451-b67f-8adada658cd6","Type":"ContainerStarted","Data":"29ca3ee7a0653dad5bf2c392a247f8f7e70bc8cdde8145ea995f97f4782887d8"} Jul 13 19:31:28.124285 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:28.124236 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-tcvzv" event={"ID":"b0f25a86-9f3a-4451-b67f-8adada658cd6","Type":"ContainerStarted","Data":"5b2fdd5a1d96f7bda7e5667abb218dcac1a6f08c0942483343c78e7e5bb4ec6d"} Jul 13 19:31:28.124704 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:28.124318 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-tcvzv" Jul 13 19:31:28.145277 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:28.145196 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-tcvzv" podStartSLOduration=1.611196753 podStartE2EDuration="4.145172122s" totalImagesPullingTime="2.533975369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:31:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:31:28.139058641 +0000 UTC m=+583.252966349" watchObservedRunningTime="2026-07-13 19:31:28.145172122 +0000 UTC m=+583.259079833" Jul 13 19:31:39.131367 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:39.131314 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-tcvzv" Jul 13 19:31:45.851059 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:45.851022 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 19:31:45.852390 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:31:45.852364 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 19:32:39.135067 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:39.135033 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:32:47.593906 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:47.593863 2556 generic.go:356] "Generic (PLEG): container finished" podID="16e74623-f867-449e-a739-4234322bb3e1" containerID="9649ab91bc4a7a6d1d862e46c3fb00159f375e8fed49a954ca0802be94b5778a" exitCode=0 Jul 13 19:32:47.594470 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:47.593938 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" event={"ID":"16e74623-f867-449e-a739-4234322bb3e1","Type":"ContainerDied","Data":"9649ab91bc4a7a6d1d862e46c3fb00159f375e8fed49a954ca0802be94b5778a"} Jul 13 19:32:47.594548 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:47.594479 2556 scope.go:117] "RemoveContainer" containerID="9649ab91bc4a7a6d1d862e46c3fb00159f375e8fed49a954ca0802be94b5778a" Jul 13 19:32:47.595637 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:47.595611 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 19:32:47.595732 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:47.595703 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 19:32:47.595786 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:47.595760 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 19:32:48.602005 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:48.601968 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" event={"ID":"16e74623-f867-449e-a739-4234322bb3e1","Type":"ContainerStarted","Data":"85385657d0b2a6d46364dbfacdc406b3cd7c5d5a4c21f3dba8bc7f34bb3083f2"} Jul 13 19:32:48.728342 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:48.728304 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:32:48.729594 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:48.729562 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-l8mnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 13 19:32:48.729741 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:48.729615 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" podUID="16e74623-f867-449e-a739-4234322bb3e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 13 19:32:49.729148 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:49.729106 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-l8mnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 13 19:32:49.729588 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:49.729174 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" podUID="16e74623-f867-449e-a739-4234322bb3e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 13 19:32:50.728769 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:50.728730 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-l8mnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 13 19:32:50.728976 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:50.728809 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" podUID="16e74623-f867-449e-a739-4234322bb3e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 13 19:32:51.728844 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:51.728771 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-l8mnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 13 19:32:51.729271 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:51.728871 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" podUID="16e74623-f867-449e-a739-4234322bb3e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 13 19:32:52.733240 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:52.733195 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:32:52.733753 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:52.733616 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:32:52.734461 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:32:52.734438 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-l8mnm" Jul 13 19:33:07.398666 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:07.398629 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:33:14.505657 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:14.505616 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:33:21.507454 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.507412 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq"] Jul 13 19:33:21.510054 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.510035 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.512167 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.512142 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 13 19:33:21.512602 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.512586 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 13 19:33:21.512653 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.512638 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4wg86\"" Jul 13 19:33:21.512763 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.512745 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 13 19:33:21.520059 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.520033 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq"] Jul 13 19:33:21.646083 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.646033 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8ad2db9-f086-4c2f-8489-b5ef9a79b944-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fnwrq\" (UID: \"b8ad2db9-f086-4c2f-8489-b5ef9a79b944\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.646299 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.646096 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8ad2db9-f086-4c2f-8489-b5ef9a79b944-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fnwrq\" (UID: \"b8ad2db9-f086-4c2f-8489-b5ef9a79b944\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.646299 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.646195 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8ad2db9-f086-4c2f-8489-b5ef9a79b944-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fnwrq\" (UID: \"b8ad2db9-f086-4c2f-8489-b5ef9a79b944\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.646299 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.646227 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8ad2db9-f086-4c2f-8489-b5ef9a79b944-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fnwrq\" (UID: \"b8ad2db9-f086-4c2f-8489-b5ef9a79b944\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.646299 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.646250 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8ad2db9-f086-4c2f-8489-b5ef9a79b944-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fnwrq\" (UID: \"b8ad2db9-f086-4c2f-8489-b5ef9a79b944\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.646539 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.646333 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h988x\" (UniqueName: \"kubernetes.io/projected/b8ad2db9-f086-4c2f-8489-b5ef9a79b944-kube-api-access-h988x\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fnwrq\" (UID: \"b8ad2db9-f086-4c2f-8489-b5ef9a79b944\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.747921 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.747882 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8ad2db9-f086-4c2f-8489-b5ef9a79b944-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fnwrq\" (UID: \"b8ad2db9-f086-4c2f-8489-b5ef9a79b944\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.748184 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.747936 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8ad2db9-f086-4c2f-8489-b5ef9a79b944-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fnwrq\" (UID: \"b8ad2db9-f086-4c2f-8489-b5ef9a79b944\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.748184 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.747959 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8ad2db9-f086-4c2f-8489-b5ef9a79b944-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fnwrq\" (UID: \"b8ad2db9-f086-4c2f-8489-b5ef9a79b944\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.748184 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.747985 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8ad2db9-f086-4c2f-8489-b5ef9a79b944-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fnwrq\" (UID: \"b8ad2db9-f086-4c2f-8489-b5ef9a79b944\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.748184 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.748024 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h988x\" (UniqueName: \"kubernetes.io/projected/b8ad2db9-f086-4c2f-8489-b5ef9a79b944-kube-api-access-h988x\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fnwrq\" (UID: \"b8ad2db9-f086-4c2f-8489-b5ef9a79b944\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.748184 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.748119 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8ad2db9-f086-4c2f-8489-b5ef9a79b944-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fnwrq\" (UID: \"b8ad2db9-f086-4c2f-8489-b5ef9a79b944\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.748470 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.748352 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8ad2db9-f086-4c2f-8489-b5ef9a79b944-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fnwrq\" (UID: \"b8ad2db9-f086-4c2f-8489-b5ef9a79b944\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.748470 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.748398 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8ad2db9-f086-4c2f-8489-b5ef9a79b944-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fnwrq\" (UID: \"b8ad2db9-f086-4c2f-8489-b5ef9a79b944\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.748592 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.748542 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8ad2db9-f086-4c2f-8489-b5ef9a79b944-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fnwrq\" (UID: \"b8ad2db9-f086-4c2f-8489-b5ef9a79b944\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.750494 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.750474 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8ad2db9-f086-4c2f-8489-b5ef9a79b944-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fnwrq\" (UID: \"b8ad2db9-f086-4c2f-8489-b5ef9a79b944\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.750767 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.750741 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8ad2db9-f086-4c2f-8489-b5ef9a79b944-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fnwrq\" (UID: \"b8ad2db9-f086-4c2f-8489-b5ef9a79b944\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.756253 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.756230 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h988x\" (UniqueName: \"kubernetes.io/projected/b8ad2db9-f086-4c2f-8489-b5ef9a79b944-kube-api-access-h988x\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fnwrq\" (UID: \"b8ad2db9-f086-4c2f-8489-b5ef9a79b944\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.821478 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.821435 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:21.971761 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.971736 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq"] Jul 13 19:33:21.972656 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:33:21.972622 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8ad2db9_f086_4c2f_8489_b5ef9a79b944.slice/crio-e1d427c598e4b87a7f68673f552d8b91a8dddf03bdd7066fe23be3042a5d5d29 WatchSource:0}: Error finding container e1d427c598e4b87a7f68673f552d8b91a8dddf03bdd7066fe23be3042a5d5d29: Status 404 returned error can't find the container with id e1d427c598e4b87a7f68673f552d8b91a8dddf03bdd7066fe23be3042a5d5d29 Jul 13 19:33:21.974168 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:21.974146 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 19:33:22.418566 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:22.418518 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:33:22.814303 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:22.814265 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" event={"ID":"b8ad2db9-f086-4c2f-8489-b5ef9a79b944","Type":"ContainerStarted","Data":"e1d427c598e4b87a7f68673f552d8b91a8dddf03bdd7066fe23be3042a5d5d29"} Jul 13 19:33:23.821999 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:23.821954 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" event={"ID":"b8ad2db9-f086-4c2f-8489-b5ef9a79b944","Type":"ContainerStarted","Data":"62503477963c94da0a3fc929949cd321f3616fc232e689e94c37d8168714394d"} Jul 13 19:33:23.842682 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:23.842606 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" podStartSLOduration=1.831295275 podStartE2EDuration="2.842583296s" totalImagesPullingTime="1.011288021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:33:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:33:23.839565062 +0000 UTC m=+698.953472771" watchObservedRunningTime="2026-07-13 19:33:23.842583296 +0000 UTC m=+698.956491000" Jul 13 19:33:30.110818 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:30.110756 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:33:31.822436 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:31.822391 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:31.822915 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:31.822492 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:31.835228 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:31.835195 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:31.879086 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:31.879055 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fnwrq" Jul 13 19:33:55.408624 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:33:55.408534 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:34:03.026413 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:34:03.026364 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:35:28.013387 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:35:28.013347 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:35:37.814441 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:35:37.814390 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:35:47.699250 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:35:47.699201 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:35:57.404186 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:35:57.404143 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:36:07.607571 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:36:07.607523 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:36:35.306339 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:36:35.306300 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:36:45.892413 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:36:45.892316 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 19:36:45.895491 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:36:45.895468 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 19:36:49.513520 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:36:49.513482 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:37:25.300116 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:37:25.300072 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:37:33.500857 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:37:33.500814 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:37:43.503752 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:37:43.503703 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:37:51.610797 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:37:51.610740 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:38:03.706471 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:38:03.706419 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:38:15.371536 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:38:15.371455 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:38:26.410188 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:38:26.410142 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:38:36.800917 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:38:36.800871 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:39:01.703290 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:39:01.703243 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:39:15.702286 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:39:15.702246 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:39:39.806202 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:39:39.806107 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:39:50.498324 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:39:50.498273 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:40:07.209200 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:40:07.209152 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:40:18.002686 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:40:18.002647 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:40:28.506041 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:40:28.506001 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:40:40.800096 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:40:40.800054 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:41:11.978551 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:11.978422 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6"] Jul 13 19:41:11.979194 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:11.978787 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6" podUID="1098758b-2aaa-4672-a073-4dd17ab700b6" containerName="manager" containerID="cri-o://482a74c42c9a97971116b670088a152778b4b79967496bc36465378131be842e" gracePeriod=10 Jul 13 19:41:16.055870 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:16.055840 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6" Jul 13 19:41:16.120374 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:16.120274 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrsld\" (UniqueName: \"kubernetes.io/projected/1098758b-2aaa-4672-a073-4dd17ab700b6-kube-api-access-mrsld\") pod \"1098758b-2aaa-4672-a073-4dd17ab700b6\" (UID: \"1098758b-2aaa-4672-a073-4dd17ab700b6\") " Jul 13 19:41:16.122343 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:16.122309 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1098758b-2aaa-4672-a073-4dd17ab700b6-kube-api-access-mrsld" (OuterVolumeSpecName: "kube-api-access-mrsld") pod "1098758b-2aaa-4672-a073-4dd17ab700b6" (UID: "1098758b-2aaa-4672-a073-4dd17ab700b6"). InnerVolumeSpecName "kube-api-access-mrsld". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:41:16.220823 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:16.220794 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1098758b-2aaa-4672-a073-4dd17ab700b6-extensions-socket-volume\") pod \"1098758b-2aaa-4672-a073-4dd17ab700b6\" (UID: \"1098758b-2aaa-4672-a073-4dd17ab700b6\") " Jul 13 19:41:16.221082 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:16.221062 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrsld\" (UniqueName: \"kubernetes.io/projected/1098758b-2aaa-4672-a073-4dd17ab700b6-kube-api-access-mrsld\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:41:16.221250 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:16.221159 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1098758b-2aaa-4672-a073-4dd17ab700b6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1098758b-2aaa-4672-a073-4dd17ab700b6" (UID: "1098758b-2aaa-4672-a073-4dd17ab700b6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:41:16.322252 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:16.322208 2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1098758b-2aaa-4672-a073-4dd17ab700b6-extensions-socket-volume\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:41:16.606581 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:16.606539 2556 generic.go:356] "Generic (PLEG): container finished" podID="1098758b-2aaa-4672-a073-4dd17ab700b6" containerID="482a74c42c9a97971116b670088a152778b4b79967496bc36465378131be842e" exitCode=0 Jul 13 19:41:16.606807 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:16.606594 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6" Jul 13 19:41:16.606807 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:16.606628 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6" event={"ID":"1098758b-2aaa-4672-a073-4dd17ab700b6","Type":"ContainerDied","Data":"482a74c42c9a97971116b670088a152778b4b79967496bc36465378131be842e"} Jul 13 19:41:16.606807 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:16.606686 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6" event={"ID":"1098758b-2aaa-4672-a073-4dd17ab700b6","Type":"ContainerDied","Data":"4e03156adf42e2cfa9461d1d5adb4cf20998d17986156cdf8e6dca87d455c77c"} Jul 13 19:41:16.606807 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:16.606718 2556 scope.go:117] "RemoveContainer" containerID="482a74c42c9a97971116b670088a152778b4b79967496bc36465378131be842e" Jul 13 19:41:16.617670 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:16.617649 2556 scope.go:117] "RemoveContainer" containerID="482a74c42c9a97971116b670088a152778b4b79967496bc36465378131be842e" Jul 13 19:41:16.618001 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:41:16.617956 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"482a74c42c9a97971116b670088a152778b4b79967496bc36465378131be842e\": container with ID starting with 482a74c42c9a97971116b670088a152778b4b79967496bc36465378131be842e not found: ID does not exist" containerID="482a74c42c9a97971116b670088a152778b4b79967496bc36465378131be842e" Jul 13 19:41:16.618068 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:16.618013 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"482a74c42c9a97971116b670088a152778b4b79967496bc36465378131be842e"} err="failed to get container status \"482a74c42c9a97971116b670088a152778b4b79967496bc36465378131be842e\": rpc error: code = NotFound desc = could not find container \"482a74c42c9a97971116b670088a152778b4b79967496bc36465378131be842e\": container with ID starting with 482a74c42c9a97971116b670088a152778b4b79967496bc36465378131be842e not found: ID does not exist" Jul 13 19:41:16.626986 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:16.626952 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6"] Jul 13 19:41:16.630514 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:16.630483 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkzg6"] Jul 13 19:41:16.849657 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:16.849618 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1098758b-2aaa-4672-a073-4dd17ab700b6" path="/var/lib/kubelet/pods/1098758b-2aaa-4672-a073-4dd17ab700b6/volumes" Jul 13 19:41:45.935057 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:45.935020 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 19:41:45.942226 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:41:45.942198 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 19:42:13.155076 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.155019 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wn6b6"] Jul 13 19:42:13.155584 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.155509 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1098758b-2aaa-4672-a073-4dd17ab700b6" containerName="manager" Jul 13 19:42:13.155584 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.155527 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1098758b-2aaa-4672-a073-4dd17ab700b6" containerName="manager" Jul 13 19:42:13.155661 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.155628 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1098758b-2aaa-4672-a073-4dd17ab700b6" containerName="manager" Jul 13 19:42:13.158972 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.158948 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wn6b6" Jul 13 19:42:13.161212 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.161190 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4vgc5\"" Jul 13 19:42:13.171459 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.171429 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wn6b6"] Jul 13 19:42:13.297657 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.297599 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05b2672c-c7b1-4b12-8418-6db7049250ce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wn6b6\" (UID: \"05b2672c-c7b1-4b12-8418-6db7049250ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wn6b6" Jul 13 19:42:13.297977 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.297709 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7pct\" (UniqueName: \"kubernetes.io/projected/05b2672c-c7b1-4b12-8418-6db7049250ce-kube-api-access-w7pct\") pod \"kuadrant-operator-controller-manager-86878c769f-wn6b6\" (UID: \"05b2672c-c7b1-4b12-8418-6db7049250ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wn6b6" Jul 13 19:42:13.398605 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.398539 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05b2672c-c7b1-4b12-8418-6db7049250ce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wn6b6\" (UID: \"05b2672c-c7b1-4b12-8418-6db7049250ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wn6b6" Jul 13 19:42:13.398605 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.398621 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7pct\" (UniqueName: \"kubernetes.io/projected/05b2672c-c7b1-4b12-8418-6db7049250ce-kube-api-access-w7pct\") pod \"kuadrant-operator-controller-manager-86878c769f-wn6b6\" (UID: \"05b2672c-c7b1-4b12-8418-6db7049250ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wn6b6" Jul 13 19:42:13.399043 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.399019 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05b2672c-c7b1-4b12-8418-6db7049250ce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wn6b6\" (UID: \"05b2672c-c7b1-4b12-8418-6db7049250ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wn6b6" Jul 13 19:42:13.407273 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.407205 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7pct\" (UniqueName: \"kubernetes.io/projected/05b2672c-c7b1-4b12-8418-6db7049250ce-kube-api-access-w7pct\") pod \"kuadrant-operator-controller-manager-86878c769f-wn6b6\" (UID: \"05b2672c-c7b1-4b12-8418-6db7049250ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wn6b6" Jul 13 19:42:13.471481 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.471438 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wn6b6" Jul 13 19:42:13.625376 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.625337 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wn6b6"] Jul 13 19:42:13.629337 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:42:13.629299 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05b2672c_c7b1_4b12_8418_6db7049250ce.slice/crio-7f7f024a464a40c7f6355b1a95176c8f278cc7463ff381926bea61a727dc7827 WatchSource:0}: Error finding container 7f7f024a464a40c7f6355b1a95176c8f278cc7463ff381926bea61a727dc7827: Status 404 returned error can't find the container with id 7f7f024a464a40c7f6355b1a95176c8f278cc7463ff381926bea61a727dc7827 Jul 13 19:42:13.941063 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.940950 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wn6b6" event={"ID":"05b2672c-c7b1-4b12-8418-6db7049250ce","Type":"ContainerStarted","Data":"e17dfa1940f30aa49066831696b43ca1fe06b3edb9cef682751de70ea8c08dbf"} Jul 13 19:42:13.941063 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.941022 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wn6b6" event={"ID":"05b2672c-c7b1-4b12-8418-6db7049250ce","Type":"ContainerStarted","Data":"7f7f024a464a40c7f6355b1a95176c8f278cc7463ff381926bea61a727dc7827"} Jul 13 19:42:13.941063 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.941057 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wn6b6" Jul 13 19:42:13.962131 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:13.962080 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wn6b6" podStartSLOduration=0.962061531 podStartE2EDuration="962.061531ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:42:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:42:13.961496472 +0000 UTC m=+1229.075404181" watchObservedRunningTime="2026-07-13 19:42:13.962061531 +0000 UTC m=+1229.075969231" Jul 13 19:42:24.949502 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:24.949468 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wn6b6" Jul 13 19:42:33.497069 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:33.497027 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:42:43.199299 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:43.199209 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:42:56.805026 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:42:56.804980 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:43:06.801978 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:43:06.801933 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:43:16.907874 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:43:16.907827 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:43:25.598402 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:43:25.598358 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:43:35.896432 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:43:35.896386 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:43:47.398764 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:43:47.398724 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:43:56.118077 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:43:56.118036 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:44:06.006482 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:44:06.006437 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:44:47.008803 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:44:47.008739 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:44:55.110670 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:44:55.110624 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:45:08.218429 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:45:08.218391 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:45:18.317427 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:45:18.317386 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:45:37.308942 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:45:37.308847 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:45:47.306434 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:45:47.306396 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:45:57.199706 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:45:57.199654 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:46:05.005034 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:46:05.004990 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:46:16.617071 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:46:16.610387 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:46:26.602612 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:46:26.602567 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:46:37.497507 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:46:37.497470 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:46:45.978133 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:46:45.978104 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 19:46:45.985193 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:46:45.985169 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 19:46:49.312743 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:46:49.312700 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:47:04.605667 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:47:04.605625 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:47:13.196168 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:47:13.196076 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:47:25.801304 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:47:25.801263 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:47:42.039377 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:47:42.039332 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:47:54.603719 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:47:54.603680 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:48:08.300608 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:08.300565 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:48:22.395045 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:22.395007 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:48:35.213608 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:35.213570 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:48:40.773105 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:40.773015 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn"] Jul 13 19:48:40.776591 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:40.776563 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:40.779124 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:40.778957 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1b0d941e-openshift-default-dockercfg-bxr2c\"" Jul 13 19:48:40.801055 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:40.801011 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn"] Jul 13 19:48:40.902442 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:40.902402 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm2qb\" (UniqueName: \"kubernetes.io/projected/af5e0fbd-5f55-4c56-af14-39c5120e4cea-kube-api-access-vm2qb\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:40.902442 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:40.902453 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istiod-ca-cert\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:40.902676 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:40.902514 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-token\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:40.902676 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:40.902564 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-credential-socket\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:40.902676 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:40.902587 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-data\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:40.902676 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:40.902644 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-workload-certs\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:40.902841 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:40.902687 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-workload-socket\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:40.902841 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:40.902709 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-podinfo\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:40.902841 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:40.902732 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-envoy\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.003807 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.003726 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm2qb\" (UniqueName: \"kubernetes.io/projected/af5e0fbd-5f55-4c56-af14-39c5120e4cea-kube-api-access-vm2qb\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.004026 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.003820 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istiod-ca-cert\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.004026 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.003851 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-token\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.004026 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.003887 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-credential-socket\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.004026 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.003914 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-data\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.004026 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.003970 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-workload-certs\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.004294 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.004034 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-workload-socket\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.004294 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.004066 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-podinfo\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.004294 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.004121 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-envoy\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.004455 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.004367 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-credential-socket\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.004455 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.004441 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-workload-certs\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.004670 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.004651 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-workload-socket\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.004670 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.004661 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-data\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.004812 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.004657 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istiod-ca-cert\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.006742 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.006718 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-envoy\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.006742 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.006732 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-podinfo\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.012808 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.012767 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-token\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.013090 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.013068 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm2qb\" (UniqueName: \"kubernetes.io/projected/af5e0fbd-5f55-4c56-af14-39c5120e4cea-kube-api-access-vm2qb\") pod \"e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.088429 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.088391 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:41.247028 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.246994 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn"] Jul 13 19:48:41.250092 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:48:41.250053 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf5e0fbd_5f55_4c56_af14_39c5120e4cea.slice/crio-abb1cfb9970b266dcb293b78fbb2c2e175bc8a8ad4f8f057fc821bd80be9b338 WatchSource:0}: Error finding container abb1cfb9970b266dcb293b78fbb2c2e175bc8a8ad4f8f057fc821bd80be9b338: Status 404 returned error can't find the container with id abb1cfb9970b266dcb293b78fbb2c2e175bc8a8ad4f8f057fc821bd80be9b338 Jul 13 19:48:41.252976 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.252939 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 19:48:41.253086 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.253035 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 19:48:41.253158 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.253082 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 19:48:41.264791 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:41.264746 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" event={"ID":"af5e0fbd-5f55-4c56-af14-39c5120e4cea","Type":"ContainerStarted","Data":"abb1cfb9970b266dcb293b78fbb2c2e175bc8a8ad4f8f057fc821bd80be9b338"} Jul 13 19:48:42.272876 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:42.272830 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" event={"ID":"af5e0fbd-5f55-4c56-af14-39c5120e4cea","Type":"ContainerStarted","Data":"111699c6076079663ac706685dda3b59b9972febd627a87c8f4aff8c6f2d71e9"} Jul 13 19:48:42.297680 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:42.297615 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" podStartSLOduration=2.297593942 podStartE2EDuration="2.297593942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:48:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:48:42.295672698 +0000 UTC m=+1617.409580425" watchObservedRunningTime="2026-07-13 19:48:42.297593942 +0000 UTC m=+1617.411501651" Jul 13 19:48:43.089539 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:43.089490 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:43.094879 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:43.094759 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:43.279447 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:43.279413 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:43.280459 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:43.280441 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:46.297024 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:46.296989 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:48:55.356950 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:55.356909 2556 generic.go:356] "Generic (PLEG): container finished" podID="af5e0fbd-5f55-4c56-af14-39c5120e4cea" containerID="111699c6076079663ac706685dda3b59b9972febd627a87c8f4aff8c6f2d71e9" exitCode=0 Jul 13 19:48:55.357519 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:55.356993 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" event={"ID":"af5e0fbd-5f55-4c56-af14-39c5120e4cea","Type":"ContainerDied","Data":"111699c6076079663ac706685dda3b59b9972febd627a87c8f4aff8c6f2d71e9"} Jul 13 19:48:55.357589 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:55.357571 2556 scope.go:117] "RemoveContainer" containerID="111699c6076079663ac706685dda3b59b9972febd627a87c8f4aff8c6f2d71e9" Jul 13 19:48:55.358981 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:55.358944 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 19:48:55.359108 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:55.359035 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 19:48:55.359108 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:55.359078 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 19:48:56.372368 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:56.372328 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" event={"ID":"af5e0fbd-5f55-4c56-af14-39c5120e4cea","Type":"ContainerStarted","Data":"8df8e0af2a7e646a1947360fb1b9bc54b5e72e26a084c70a532ab6ac75ed3dd5"} Jul 13 19:48:57.089243 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:57.089192 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:57.090556 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:57.090533 2556 patch_prober.go:28] interesting pod/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 13 19:48:57.090672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:57.090584 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" podUID="af5e0fbd-5f55-4c56-af14-39c5120e4cea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 13 19:48:58.089919 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:58.089880 2556 patch_prober.go:28] interesting pod/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 13 19:48:58.090298 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:58.089939 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" podUID="af5e0fbd-5f55-4c56-af14-39c5120e4cea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 13 19:48:58.280005 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:58.279944 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:48:59.089204 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:59.089160 2556 patch_prober.go:28] interesting pod/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 13 19:48:59.089410 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:48:59.089225 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" podUID="af5e0fbd-5f55-4c56-af14-39c5120e4cea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 13 19:49:00.089432 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:49:00.089390 2556 patch_prober.go:28] interesting pod/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 13 19:49:00.089888 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:49:00.089456 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" podUID="af5e0fbd-5f55-4c56-af14-39c5120e4cea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 13 19:49:01.093075 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:49:01.093044 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:49:01.093997 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:49:01.093974 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:50:00.503335 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:00.503302 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:50:12.303536 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:12.303443 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn"] Jul 13 19:50:12.303981 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:12.303664 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" podUID="af5e0fbd-5f55-4c56-af14-39c5120e4cea" containerName="istio-proxy" containerID="cri-o://8df8e0af2a7e646a1947360fb1b9bc54b5e72e26a084c70a532ab6ac75ed3dd5" gracePeriod=30 Jul 13 19:50:16.095464 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:16.095422 2556 patch_prober.go:28] interesting pod/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 19:50:16.095922 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:16.095499 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" podUID="af5e0fbd-5f55-4c56-af14-39c5120e4cea" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 19:50:17.574610 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.574530 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:50:17.603951 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.603906 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-workload-certs\") pod \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " Jul 13 19:50:17.604154 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.603967 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-credential-socket\") pod \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " Jul 13 19:50:17.604154 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.604049 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-data\") pod \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " Jul 13 19:50:17.604154 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.604126 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-workload-socket\") pod \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " Jul 13 19:50:17.604336 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.604169 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-token\") pod \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " Jul 13 19:50:17.604336 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.604206 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istiod-ca-cert\") pod \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " Jul 13 19:50:17.604336 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.604231 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "af5e0fbd-5f55-4c56-af14-39c5120e4cea" (UID: "af5e0fbd-5f55-4c56-af14-39c5120e4cea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:50:17.604336 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.604254 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vm2qb\" (UniqueName: \"kubernetes.io/projected/af5e0fbd-5f55-4c56-af14-39c5120e4cea-kube-api-access-vm2qb\") pod \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " Jul 13 19:50:17.604544 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.604335 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-envoy\") pod \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " Jul 13 19:50:17.604544 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.604373 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-podinfo\") pod \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\" (UID: \"af5e0fbd-5f55-4c56-af14-39c5120e4cea\") " Jul 13 19:50:17.604670 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.604619 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "af5e0fbd-5f55-4c56-af14-39c5120e4cea" (UID: "af5e0fbd-5f55-4c56-af14-39c5120e4cea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:50:17.605321 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.605208 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "af5e0fbd-5f55-4c56-af14-39c5120e4cea" (UID: "af5e0fbd-5f55-4c56-af14-39c5120e4cea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:50:17.605321 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.605245 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-workload-socket\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:50:17.605321 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.605264 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-workload-certs\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:50:17.605538 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.605433 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "af5e0fbd-5f55-4c56-af14-39c5120e4cea" (UID: "af5e0fbd-5f55-4c56-af14-39c5120e4cea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:50:17.606077 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.606024 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-data" (OuterVolumeSpecName: "istio-data") pod "af5e0fbd-5f55-4c56-af14-39c5120e4cea" (UID: "af5e0fbd-5f55-4c56-af14-39c5120e4cea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:50:17.606950 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.606909 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-token" (OuterVolumeSpecName: "istio-token") pod "af5e0fbd-5f55-4c56-af14-39c5120e4cea" (UID: "af5e0fbd-5f55-4c56-af14-39c5120e4cea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:50:17.607737 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.607712 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "af5e0fbd-5f55-4c56-af14-39c5120e4cea" (UID: "af5e0fbd-5f55-4c56-af14-39c5120e4cea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 19:50:17.607967 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.607733 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af5e0fbd-5f55-4c56-af14-39c5120e4cea-kube-api-access-vm2qb" (OuterVolumeSpecName: "kube-api-access-vm2qb") pod "af5e0fbd-5f55-4c56-af14-39c5120e4cea" (UID: "af5e0fbd-5f55-4c56-af14-39c5120e4cea"). InnerVolumeSpecName "kube-api-access-vm2qb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:50:17.608018 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.607975 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "af5e0fbd-5f55-4c56-af14-39c5120e4cea" (UID: "af5e0fbd-5f55-4c56-af14-39c5120e4cea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:50:17.706637 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.706605 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-token\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:50:17.706637 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.706634 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istiod-ca-cert\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:50:17.706637 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.706644 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vm2qb\" (UniqueName: \"kubernetes.io/projected/af5e0fbd-5f55-4c56-af14-39c5120e4cea-kube-api-access-vm2qb\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:50:17.706884 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.706655 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-envoy\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:50:17.706884 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.706664 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-podinfo\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:50:17.706884 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.706672 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-credential-socket\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:50:17.706884 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.706680 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af5e0fbd-5f55-4c56-af14-39c5120e4cea-istio-data\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:50:17.855591 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.855495 2556 generic.go:356] "Generic (PLEG): container finished" podID="af5e0fbd-5f55-4c56-af14-39c5120e4cea" containerID="8df8e0af2a7e646a1947360fb1b9bc54b5e72e26a084c70a532ab6ac75ed3dd5" exitCode=0 Jul 13 19:50:17.855591 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.855553 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" Jul 13 19:50:17.855591 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.855574 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" event={"ID":"af5e0fbd-5f55-4c56-af14-39c5120e4cea","Type":"ContainerDied","Data":"8df8e0af2a7e646a1947360fb1b9bc54b5e72e26a084c70a532ab6ac75ed3dd5"} Jul 13 19:50:17.855870 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.855631 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn" event={"ID":"af5e0fbd-5f55-4c56-af14-39c5120e4cea","Type":"ContainerDied","Data":"abb1cfb9970b266dcb293b78fbb2c2e175bc8a8ad4f8f057fc821bd80be9b338"} Jul 13 19:50:17.855870 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.855655 2556 scope.go:117] "RemoveContainer" containerID="8df8e0af2a7e646a1947360fb1b9bc54b5e72e26a084c70a532ab6ac75ed3dd5" Jul 13 19:50:17.870492 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.870468 2556 scope.go:117] "RemoveContainer" containerID="111699c6076079663ac706685dda3b59b9972febd627a87c8f4aff8c6f2d71e9" Jul 13 19:50:17.880099 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.880067 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn"] Jul 13 19:50:17.881828 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.881807 2556 scope.go:117] "RemoveContainer" containerID="8df8e0af2a7e646a1947360fb1b9bc54b5e72e26a084c70a532ab6ac75ed3dd5" Jul 13 19:50:17.882193 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:50:17.882175 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8df8e0af2a7e646a1947360fb1b9bc54b5e72e26a084c70a532ab6ac75ed3dd5\": container with ID starting with 8df8e0af2a7e646a1947360fb1b9bc54b5e72e26a084c70a532ab6ac75ed3dd5 not found: ID does not exist" containerID="8df8e0af2a7e646a1947360fb1b9bc54b5e72e26a084c70a532ab6ac75ed3dd5" Jul 13 19:50:17.882238 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.882204 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8df8e0af2a7e646a1947360fb1b9bc54b5e72e26a084c70a532ab6ac75ed3dd5"} err="failed to get container status \"8df8e0af2a7e646a1947360fb1b9bc54b5e72e26a084c70a532ab6ac75ed3dd5\": rpc error: code = NotFound desc = could not find container \"8df8e0af2a7e646a1947360fb1b9bc54b5e72e26a084c70a532ab6ac75ed3dd5\": container with ID starting with 8df8e0af2a7e646a1947360fb1b9bc54b5e72e26a084c70a532ab6ac75ed3dd5 not found: ID does not exist" Jul 13 19:50:17.882238 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.882224 2556 scope.go:117] "RemoveContainer" containerID="111699c6076079663ac706685dda3b59b9972febd627a87c8f4aff8c6f2d71e9" Jul 13 19:50:17.882519 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:50:17.882498 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"111699c6076079663ac706685dda3b59b9972febd627a87c8f4aff8c6f2d71e9\": container with ID starting with 111699c6076079663ac706685dda3b59b9972febd627a87c8f4aff8c6f2d71e9 not found: ID does not exist" containerID="111699c6076079663ac706685dda3b59b9972febd627a87c8f4aff8c6f2d71e9" Jul 13 19:50:17.882562 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.882525 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"111699c6076079663ac706685dda3b59b9972febd627a87c8f4aff8c6f2d71e9"} err="failed to get container status \"111699c6076079663ac706685dda3b59b9972febd627a87c8f4aff8c6f2d71e9\": rpc error: code = NotFound desc = could not find container \"111699c6076079663ac706685dda3b59b9972febd627a87c8f4aff8c6f2d71e9\": container with ID starting with 111699c6076079663ac706685dda3b59b9972febd627a87c8f4aff8c6f2d71e9 not found: ID does not exist" Jul 13 19:50:17.882690 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:17.882674 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1b0d941e-openshift-default-79578cbbdc-ggpvn"] Jul 13 19:50:18.850095 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:18.850057 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af5e0fbd-5f55-4c56-af14-39c5120e4cea" path="/var/lib/kubelet/pods/af5e0fbd-5f55-4c56-af14-39c5120e4cea/volumes" Jul 13 19:50:21.812253 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:50:21.812210 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:51:21.906474 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:21.906429 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:51:32.228641 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.228587 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c"] Jul 13 19:51:32.229083 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.229054 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af5e0fbd-5f55-4c56-af14-39c5120e4cea" containerName="istio-proxy" Jul 13 19:51:32.229083 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.229067 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="af5e0fbd-5f55-4c56-af14-39c5120e4cea" containerName="istio-proxy" Jul 13 19:51:32.229083 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.229077 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af5e0fbd-5f55-4c56-af14-39c5120e4cea" containerName="istio-proxy" Jul 13 19:51:32.229083 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.229083 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="af5e0fbd-5f55-4c56-af14-39c5120e4cea" containerName="istio-proxy" Jul 13 19:51:32.229219 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.229167 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="af5e0fbd-5f55-4c56-af14-39c5120e4cea" containerName="istio-proxy" Jul 13 19:51:32.229219 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.229182 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="af5e0fbd-5f55-4c56-af14-39c5120e4cea" containerName="istio-proxy" Jul 13 19:51:32.232746 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.232717 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" Jul 13 19:51:32.235280 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.235255 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 19:51:32.235627 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.235602 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-32278165-serving-cert\"" Jul 13 19:51:32.236253 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.236232 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 19:51:32.236836 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.236818 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q6tkl\"" Jul 13 19:51:32.247377 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.247344 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c"] Jul 13 19:51:32.360269 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.360225 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb-openshift-service-ca\") pod \"maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c\" (UID: \"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" Jul 13 19:51:32.360492 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.360334 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb-maas-api-tls\") pod \"maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c\" (UID: \"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" Jul 13 19:51:32.360492 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.360450 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-966cp\" (UniqueName: \"kubernetes.io/projected/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb-kube-api-access-966cp\") pod \"maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c\" (UID: \"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" Jul 13 19:51:32.462692 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.461629 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-966cp\" (UniqueName: \"kubernetes.io/projected/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb-kube-api-access-966cp\") pod \"maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c\" (UID: \"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" Jul 13 19:51:32.462692 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.461701 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb-openshift-service-ca\") pod \"maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c\" (UID: \"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" Jul 13 19:51:32.462692 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.461825 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb-maas-api-tls\") pod \"maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c\" (UID: \"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" Jul 13 19:51:32.462692 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.462608 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb-openshift-service-ca\") pod \"maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c\" (UID: \"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" Jul 13 19:51:32.465255 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.465228 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb-maas-api-tls\") pod \"maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c\" (UID: \"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" Jul 13 19:51:32.474982 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.474944 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-966cp\" (UniqueName: \"kubernetes.io/projected/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb-kube-api-access-966cp\") pod \"maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c\" (UID: \"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" Jul 13 19:51:32.547825 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.547786 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" Jul 13 19:51:32.705317 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.705284 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c"] Jul 13 19:51:32.707052 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:51:32.707019 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ffe75ab_20d4_4fde_b398_7ae6b7f9f3bb.slice/crio-3d07af73440b6018678df8d9966e164d558c3b184496e820cb244e095863e442 WatchSource:0}: Error finding container 3d07af73440b6018678df8d9966e164d558c3b184496e820cb244e095863e442: Status 404 returned error can't find the container with id 3d07af73440b6018678df8d9966e164d558c3b184496e820cb244e095863e442 Jul 13 19:51:32.708563 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:32.708545 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 19:51:33.302310 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:33.302268 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" event={"ID":"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb","Type":"ContainerStarted","Data":"3d07af73440b6018678df8d9966e164d558c3b184496e820cb244e095863e442"} Jul 13 19:51:35.315635 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:35.315522 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" event={"ID":"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb","Type":"ContainerStarted","Data":"087b9df55578e8cde18cbff4974504eab373e76a01989cb29a60177f0a8a86a9"} Jul 13 19:51:35.315635 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:35.315604 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" Jul 13 19:51:35.337361 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:35.337307 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" podStartSLOduration=0.991088165 podStartE2EDuration="3.337290496s" totalImagesPullingTime="2.346202331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:51:35.33561823 +0000 UTC m=+1790.449525942" watchObservedRunningTime="2026-07-13 19:51:35.337290496 +0000 UTC m=+1790.451198204" Jul 13 19:51:35.513109 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:35.513065 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:51:41.330003 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:41.329911 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" Jul 13 19:51:46.019303 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:46.019265 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 19:51:46.026296 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:51:46.026269 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 19:52:47.656084 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:47.656048 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c"] Jul 13 19:52:47.656663 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:47.656285 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" podUID="1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb" containerName="maas-api" containerID="cri-o://087b9df55578e8cde18cbff4974504eab373e76a01989cb29a60177f0a8a86a9" gracePeriod=30 Jul 13 19:52:47.932906 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:47.932871 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" Jul 13 19:52:48.002135 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.002097 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb-maas-api-tls\") pod \"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb\" (UID: \"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb\") " Jul 13 19:52:48.002331 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.002148 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb-openshift-service-ca\") pod \"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb\" (UID: \"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb\") " Jul 13 19:52:48.002331 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.002290 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-966cp\" (UniqueName: \"kubernetes.io/projected/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb-kube-api-access-966cp\") pod \"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb\" (UID: \"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb\") " Jul 13 19:52:48.002568 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.002543 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb" (UID: "1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:52:48.002672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.002656 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb-openshift-service-ca\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:52:48.004600 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.004573 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb-kube-api-access-966cp" (OuterVolumeSpecName: "kube-api-access-966cp") pod "1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb" (UID: "1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb"). InnerVolumeSpecName "kube-api-access-966cp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:52:48.004600 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.004592 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb" (UID: "1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:52:48.103749 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.103700 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb-maas-api-tls\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:52:48.103749 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.103740 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-966cp\" (UniqueName: \"kubernetes.io/projected/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb-kube-api-access-966cp\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:52:48.774728 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.774691 2556 generic.go:356] "Generic (PLEG): container finished" podID="1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb" containerID="087b9df55578e8cde18cbff4974504eab373e76a01989cb29a60177f0a8a86a9" exitCode=0 Jul 13 19:52:48.774728 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.774733 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" event={"ID":"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb","Type":"ContainerDied","Data":"087b9df55578e8cde18cbff4974504eab373e76a01989cb29a60177f0a8a86a9"} Jul 13 19:52:48.775241 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.774758 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" Jul 13 19:52:48.775241 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.774805 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c" event={"ID":"1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb","Type":"ContainerDied","Data":"3d07af73440b6018678df8d9966e164d558c3b184496e820cb244e095863e442"} Jul 13 19:52:48.775241 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.774830 2556 scope.go:117] "RemoveContainer" containerID="087b9df55578e8cde18cbff4974504eab373e76a01989cb29a60177f0a8a86a9" Jul 13 19:52:48.785264 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.785241 2556 scope.go:117] "RemoveContainer" containerID="087b9df55578e8cde18cbff4974504eab373e76a01989cb29a60177f0a8a86a9" Jul 13 19:52:48.785616 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:52:48.785590 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"087b9df55578e8cde18cbff4974504eab373e76a01989cb29a60177f0a8a86a9\": container with ID starting with 087b9df55578e8cde18cbff4974504eab373e76a01989cb29a60177f0a8a86a9 not found: ID does not exist" containerID="087b9df55578e8cde18cbff4974504eab373e76a01989cb29a60177f0a8a86a9" Jul 13 19:52:48.785709 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.785622 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"087b9df55578e8cde18cbff4974504eab373e76a01989cb29a60177f0a8a86a9"} err="failed to get container status \"087b9df55578e8cde18cbff4974504eab373e76a01989cb29a60177f0a8a86a9\": rpc error: code = NotFound desc = could not find container \"087b9df55578e8cde18cbff4974504eab373e76a01989cb29a60177f0a8a86a9\": container with ID starting with 087b9df55578e8cde18cbff4974504eab373e76a01989cb29a60177f0a8a86a9 not found: ID does not exist" Jul 13 19:52:48.798202 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.798162 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c"] Jul 13 19:52:48.801808 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.801761 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32278165-5dcf6d5666-ngb7c"] Jul 13 19:52:48.852040 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:48.852005 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb" path="/var/lib/kubelet/pods/1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb/volumes" Jul 13 19:52:50.605672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:52:50.605639 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:53:04.995457 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:04.995414 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f"] Jul 13 19:53:04.996113 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:04.996090 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb" containerName="maas-api" Jul 13 19:53:04.996207 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:04.996117 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb" containerName="maas-api" Jul 13 19:53:04.996272 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:04.996257 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ffe75ab-20d4-4fde-b398-7ae6b7f9f3bb" containerName="maas-api" Jul 13 19:53:05.001645 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:05.001611 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" Jul 13 19:53:05.003965 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:05.003934 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 19:53:05.003965 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:05.003958 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q6tkl\"" Jul 13 19:53:05.004203 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:05.004115 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 19:53:05.004360 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:05.004336 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-7a28704c-serving-cert\"" Jul 13 19:53:05.011614 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:05.011407 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f"] Jul 13 19:53:05.079801 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:05.079726 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f32f828a-0aba-4ddf-9cd7-04193d5692a1-maas-api-tls\") pod \"maas-api-e2e-derive-7a28704c-64765b9459-46f7f\" (UID: \"f32f828a-0aba-4ddf-9cd7-04193d5692a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" Jul 13 19:53:05.080005 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:05.079842 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb5td\" (UniqueName: \"kubernetes.io/projected/f32f828a-0aba-4ddf-9cd7-04193d5692a1-kube-api-access-vb5td\") pod \"maas-api-e2e-derive-7a28704c-64765b9459-46f7f\" (UID: \"f32f828a-0aba-4ddf-9cd7-04193d5692a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" Jul 13 19:53:05.080005 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:05.079892 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f32f828a-0aba-4ddf-9cd7-04193d5692a1-openshift-service-ca\") pod \"maas-api-e2e-derive-7a28704c-64765b9459-46f7f\" (UID: \"f32f828a-0aba-4ddf-9cd7-04193d5692a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" Jul 13 19:53:05.181410 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:05.181371 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f32f828a-0aba-4ddf-9cd7-04193d5692a1-maas-api-tls\") pod \"maas-api-e2e-derive-7a28704c-64765b9459-46f7f\" (UID: \"f32f828a-0aba-4ddf-9cd7-04193d5692a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" Jul 13 19:53:05.181619 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:05.181424 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb5td\" (UniqueName: \"kubernetes.io/projected/f32f828a-0aba-4ddf-9cd7-04193d5692a1-kube-api-access-vb5td\") pod \"maas-api-e2e-derive-7a28704c-64765b9459-46f7f\" (UID: \"f32f828a-0aba-4ddf-9cd7-04193d5692a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" Jul 13 19:53:05.181619 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:05.181466 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f32f828a-0aba-4ddf-9cd7-04193d5692a1-openshift-service-ca\") pod \"maas-api-e2e-derive-7a28704c-64765b9459-46f7f\" (UID: \"f32f828a-0aba-4ddf-9cd7-04193d5692a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" Jul 13 19:53:05.182171 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:05.182148 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f32f828a-0aba-4ddf-9cd7-04193d5692a1-openshift-service-ca\") pod \"maas-api-e2e-derive-7a28704c-64765b9459-46f7f\" (UID: \"f32f828a-0aba-4ddf-9cd7-04193d5692a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" Jul 13 19:53:05.183959 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:05.183938 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f32f828a-0aba-4ddf-9cd7-04193d5692a1-maas-api-tls\") pod \"maas-api-e2e-derive-7a28704c-64765b9459-46f7f\" (UID: \"f32f828a-0aba-4ddf-9cd7-04193d5692a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" Jul 13 19:53:05.192963 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:05.192927 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb5td\" (UniqueName: \"kubernetes.io/projected/f32f828a-0aba-4ddf-9cd7-04193d5692a1-kube-api-access-vb5td\") pod \"maas-api-e2e-derive-7a28704c-64765b9459-46f7f\" (UID: \"f32f828a-0aba-4ddf-9cd7-04193d5692a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" Jul 13 19:53:05.317596 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:05.317545 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" Jul 13 19:53:05.468963 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:05.468929 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f"] Jul 13 19:53:05.471535 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:53:05.471500 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf32f828a_0aba_4ddf_9cd7_04193d5692a1.slice/crio-169b834f8abe1bef41352016d956bd806d8dedbb3c972839231fbdd8a9217d42 WatchSource:0}: Error finding container 169b834f8abe1bef41352016d956bd806d8dedbb3c972839231fbdd8a9217d42: Status 404 returned error can't find the container with id 169b834f8abe1bef41352016d956bd806d8dedbb3c972839231fbdd8a9217d42 Jul 13 19:53:05.881505 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:05.881461 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" event={"ID":"f32f828a-0aba-4ddf-9cd7-04193d5692a1","Type":"ContainerStarted","Data":"169b834f8abe1bef41352016d956bd806d8dedbb3c972839231fbdd8a9217d42"} Jul 13 19:53:06.890553 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:06.890455 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" event={"ID":"f32f828a-0aba-4ddf-9cd7-04193d5692a1","Type":"ContainerStarted","Data":"55d8c9bcb955a471fb7c10c6ebed3f4b469d20746d17fed315e82a7cb47b23e5"} Jul 13 19:53:06.890553 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:06.890525 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" Jul 13 19:53:06.912427 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:06.912347 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" podStartSLOduration=2.441310926 podStartE2EDuration="2.912321796s" totalImagesPullingTime="471.01087ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:53:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:53:06.906066651 +0000 UTC m=+1882.019974373" watchObservedRunningTime="2026-07-13 19:53:06.912321796 +0000 UTC m=+1882.026229508" Jul 13 19:53:12.797074 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:12.797028 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:53:12.903298 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:53:12.903265 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" Jul 13 19:54:19.679927 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:19.679881 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f"] Jul 13 19:54:19.683378 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:19.680709 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" podUID="f32f828a-0aba-4ddf-9cd7-04193d5692a1" containerName="maas-api" containerID="cri-o://55d8c9bcb955a471fb7c10c6ebed3f4b469d20746d17fed315e82a7cb47b23e5" gracePeriod=30 Jul 13 19:54:19.949672 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:19.949634 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" Jul 13 19:54:20.052261 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.052224 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f32f828a-0aba-4ddf-9cd7-04193d5692a1-maas-api-tls\") pod \"f32f828a-0aba-4ddf-9cd7-04193d5692a1\" (UID: \"f32f828a-0aba-4ddf-9cd7-04193d5692a1\") " Jul 13 19:54:20.052471 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.052279 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f32f828a-0aba-4ddf-9cd7-04193d5692a1-openshift-service-ca\") pod \"f32f828a-0aba-4ddf-9cd7-04193d5692a1\" (UID: \"f32f828a-0aba-4ddf-9cd7-04193d5692a1\") " Jul 13 19:54:20.052471 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.052394 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vb5td\" (UniqueName: \"kubernetes.io/projected/f32f828a-0aba-4ddf-9cd7-04193d5692a1-kube-api-access-vb5td\") pod \"f32f828a-0aba-4ddf-9cd7-04193d5692a1\" (UID: \"f32f828a-0aba-4ddf-9cd7-04193d5692a1\") " Jul 13 19:54:20.052678 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.052652 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f32f828a-0aba-4ddf-9cd7-04193d5692a1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f32f828a-0aba-4ddf-9cd7-04193d5692a1" (UID: "f32f828a-0aba-4ddf-9cd7-04193d5692a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:54:20.054606 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.054579 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f32f828a-0aba-4ddf-9cd7-04193d5692a1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f32f828a-0aba-4ddf-9cd7-04193d5692a1" (UID: "f32f828a-0aba-4ddf-9cd7-04193d5692a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:54:20.054760 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.054736 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f32f828a-0aba-4ddf-9cd7-04193d5692a1-kube-api-access-vb5td" (OuterVolumeSpecName: "kube-api-access-vb5td") pod "f32f828a-0aba-4ddf-9cd7-04193d5692a1" (UID: "f32f828a-0aba-4ddf-9cd7-04193d5692a1"). InnerVolumeSpecName "kube-api-access-vb5td". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:54:20.153157 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.153114 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vb5td\" (UniqueName: \"kubernetes.io/projected/f32f828a-0aba-4ddf-9cd7-04193d5692a1-kube-api-access-vb5td\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:54:20.153157 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.153150 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f32f828a-0aba-4ddf-9cd7-04193d5692a1-maas-api-tls\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:54:20.153157 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.153165 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f32f828a-0aba-4ddf-9cd7-04193d5692a1-openshift-service-ca\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:54:20.342668 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.342628 2556 generic.go:356] "Generic (PLEG): container finished" podID="f32f828a-0aba-4ddf-9cd7-04193d5692a1" containerID="55d8c9bcb955a471fb7c10c6ebed3f4b469d20746d17fed315e82a7cb47b23e5" exitCode=0 Jul 13 19:54:20.343004 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.342685 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" Jul 13 19:54:20.343004 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.342719 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" event={"ID":"f32f828a-0aba-4ddf-9cd7-04193d5692a1","Type":"ContainerDied","Data":"55d8c9bcb955a471fb7c10c6ebed3f4b469d20746d17fed315e82a7cb47b23e5"} Jul 13 19:54:20.343004 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.342804 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f" event={"ID":"f32f828a-0aba-4ddf-9cd7-04193d5692a1","Type":"ContainerDied","Data":"169b834f8abe1bef41352016d956bd806d8dedbb3c972839231fbdd8a9217d42"} Jul 13 19:54:20.343004 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.342829 2556 scope.go:117] "RemoveContainer" containerID="55d8c9bcb955a471fb7c10c6ebed3f4b469d20746d17fed315e82a7cb47b23e5" Jul 13 19:54:20.354799 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.354758 2556 scope.go:117] "RemoveContainer" containerID="55d8c9bcb955a471fb7c10c6ebed3f4b469d20746d17fed315e82a7cb47b23e5" Jul 13 19:54:20.355145 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:54:20.355123 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55d8c9bcb955a471fb7c10c6ebed3f4b469d20746d17fed315e82a7cb47b23e5\": container with ID starting with 55d8c9bcb955a471fb7c10c6ebed3f4b469d20746d17fed315e82a7cb47b23e5 not found: ID does not exist" containerID="55d8c9bcb955a471fb7c10c6ebed3f4b469d20746d17fed315e82a7cb47b23e5" Jul 13 19:54:20.355220 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.355154 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55d8c9bcb955a471fb7c10c6ebed3f4b469d20746d17fed315e82a7cb47b23e5"} err="failed to get container status \"55d8c9bcb955a471fb7c10c6ebed3f4b469d20746d17fed315e82a7cb47b23e5\": rpc error: code = NotFound desc = could not find container \"55d8c9bcb955a471fb7c10c6ebed3f4b469d20746d17fed315e82a7cb47b23e5\": container with ID starting with 55d8c9bcb955a471fb7c10c6ebed3f4b469d20746d17fed315e82a7cb47b23e5 not found: ID does not exist" Jul 13 19:54:20.366611 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.366562 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f"] Jul 13 19:54:20.369780 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.369741 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7a28704c-64765b9459-46f7f"] Jul 13 19:54:20.851264 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:20.851225 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f32f828a-0aba-4ddf-9cd7-04193d5692a1" path="/var/lib/kubelet/pods/f32f828a-0aba-4ddf-9cd7-04193d5692a1/volumes" Jul 13 19:54:23.002149 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:54:23.002111 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:55:58.009490 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:55:58.009438 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:56:07.999020 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:56:07.998930 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:56:20.102129 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:56:20.102088 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:56:41.000851 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:56:41.000799 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:56:46.059532 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:56:46.059497 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 19:56:46.068953 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:56:46.068922 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 19:57:15.847280 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:15.847201 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt"] Jul 13 19:57:15.847955 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:15.847935 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f32f828a-0aba-4ddf-9cd7-04193d5692a1" containerName="maas-api" Jul 13 19:57:15.848026 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:15.847973 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f32f828a-0aba-4ddf-9cd7-04193d5692a1" containerName="maas-api" Jul 13 19:57:15.848106 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:15.848093 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f32f828a-0aba-4ddf-9cd7-04193d5692a1" containerName="maas-api" Jul 13 19:57:15.851899 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:15.851876 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" Jul 13 19:57:15.854313 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:15.854220 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-d67d76ca-serving-cert\"" Jul 13 19:57:15.854313 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:15.854271 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q6tkl\"" Jul 13 19:57:15.854313 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:15.854285 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 19:57:15.854570 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:15.854282 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 19:57:15.862378 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:15.862346 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt"] Jul 13 19:57:15.935977 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:15.935933 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ced9247-d576-4280-b36e-4af381ff6f3d-openshift-service-ca\") pod \"maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt\" (UID: \"9ced9247-d576-4280-b36e-4af381ff6f3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" Jul 13 19:57:15.936209 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:15.935989 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ced9247-d576-4280-b36e-4af381ff6f3d-maas-api-tls\") pod \"maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt\" (UID: \"9ced9247-d576-4280-b36e-4af381ff6f3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" Jul 13 19:57:15.936209 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:15.936113 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4bm5\" (UniqueName: \"kubernetes.io/projected/9ced9247-d576-4280-b36e-4af381ff6f3d-kube-api-access-l4bm5\") pod \"maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt\" (UID: \"9ced9247-d576-4280-b36e-4af381ff6f3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" Jul 13 19:57:16.036998 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:16.036956 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4bm5\" (UniqueName: \"kubernetes.io/projected/9ced9247-d576-4280-b36e-4af381ff6f3d-kube-api-access-l4bm5\") pod \"maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt\" (UID: \"9ced9247-d576-4280-b36e-4af381ff6f3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" Jul 13 19:57:16.037218 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:16.037040 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ced9247-d576-4280-b36e-4af381ff6f3d-openshift-service-ca\") pod \"maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt\" (UID: \"9ced9247-d576-4280-b36e-4af381ff6f3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" Jul 13 19:57:16.037218 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:16.037076 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ced9247-d576-4280-b36e-4af381ff6f3d-maas-api-tls\") pod \"maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt\" (UID: \"9ced9247-d576-4280-b36e-4af381ff6f3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" Jul 13 19:57:16.037218 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:57:16.037207 2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-serving-cert: secret "maas-api-e2e-mt-d67d76ca-serving-cert" not found Jul 13 19:57:16.037380 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:57:16.037282 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ced9247-d576-4280-b36e-4af381ff6f3d-maas-api-tls podName:9ced9247-d576-4280-b36e-4af381ff6f3d nodeName:}" failed. No retries permitted until 2026-07-13 19:57:16.537259697 +0000 UTC m=+2131.651167384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9ced9247-d576-4280-b36e-4af381ff6f3d-maas-api-tls") pod "maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" (UID: "9ced9247-d576-4280-b36e-4af381ff6f3d") : secret "maas-api-e2e-mt-d67d76ca-serving-cert" not found Jul 13 19:57:16.037815 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:16.037768 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ced9247-d576-4280-b36e-4af381ff6f3d-openshift-service-ca\") pod \"maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt\" (UID: \"9ced9247-d576-4280-b36e-4af381ff6f3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" Jul 13 19:57:16.057196 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:16.057161 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4bm5\" (UniqueName: \"kubernetes.io/projected/9ced9247-d576-4280-b36e-4af381ff6f3d-kube-api-access-l4bm5\") pod \"maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt\" (UID: \"9ced9247-d576-4280-b36e-4af381ff6f3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" Jul 13 19:57:16.541954 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:16.541913 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ced9247-d576-4280-b36e-4af381ff6f3d-maas-api-tls\") pod \"maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt\" (UID: \"9ced9247-d576-4280-b36e-4af381ff6f3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" Jul 13 19:57:16.545455 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:16.545415 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ced9247-d576-4280-b36e-4af381ff6f3d-maas-api-tls\") pod \"maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt\" (UID: \"9ced9247-d576-4280-b36e-4af381ff6f3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" Jul 13 19:57:16.766229 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:16.766185 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" Jul 13 19:57:16.916929 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:16.916901 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt"] Jul 13 19:57:16.919487 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:57:16.919451 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ced9247_d576_4280_b36e_4af381ff6f3d.slice/crio-43d711a89a6871ced4b9dcac85c9a95588ed7e8453c85fd81d8b2daee55365d5 WatchSource:0}: Error finding container 43d711a89a6871ced4b9dcac85c9a95588ed7e8453c85fd81d8b2daee55365d5: Status 404 returned error can't find the container with id 43d711a89a6871ced4b9dcac85c9a95588ed7e8453c85fd81d8b2daee55365d5 Jul 13 19:57:16.921198 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:16.921176 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 19:57:17.404564 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:17.404535 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" event={"ID":"9ced9247-d576-4280-b36e-4af381ff6f3d","Type":"ContainerStarted","Data":"43d711a89a6871ced4b9dcac85c9a95588ed7e8453c85fd81d8b2daee55365d5"} Jul 13 19:57:18.411994 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:18.411948 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" event={"ID":"9ced9247-d576-4280-b36e-4af381ff6f3d","Type":"ContainerStarted","Data":"d03a9aafe78b50a99a3fda1f570041296761beb266d8256d0046449932df6dd4"} Jul 13 19:57:18.412443 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:18.412024 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" Jul 13 19:57:18.430907 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:18.430844 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" podStartSLOduration=2.95081675 podStartE2EDuration="3.430824238s" totalImagesPullingTime="480.007488ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:57:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:57:18.428058842 +0000 UTC m=+2133.541966553" watchObservedRunningTime="2026-07-13 19:57:18.430824238 +0000 UTC m=+2133.544731949" Jul 13 19:57:21.406797 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:21.406744 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:57:24.423081 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:24.423049 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" Jul 13 19:57:33.504193 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:33.504152 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:57:58.102072 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:57:58.101978 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:58:04.004832 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:04.004767 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j"] Jul 13 19:58:04.008671 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:04.008649 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j" Jul 13 19:58:04.010726 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:04.010706 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q5n9g\"" Jul 13 19:58:04.015617 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:04.015593 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j"] Jul 13 19:58:04.037971 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:04.037935 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40814be3-8200-431a-a592-c1bfe7cb53c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j\" (UID: \"40814be3-8200-431a-a592-c1bfe7cb53c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j" Jul 13 19:58:04.138918 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:04.138879 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40814be3-8200-431a-a592-c1bfe7cb53c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j\" (UID: \"40814be3-8200-431a-a592-c1bfe7cb53c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j" Jul 13 19:58:04.139588 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:04.139568 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40814be3-8200-431a-a592-c1bfe7cb53c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j\" (UID: \"40814be3-8200-431a-a592-c1bfe7cb53c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j" Jul 13 19:58:04.322034 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:04.321998 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j" Jul 13 19:58:04.468307 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:04.468271 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j"] Jul 13 19:58:04.469968 ip-10-0-132-20 kubenswrapper[2556]: W0713 19:58:04.469938 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40814be3_8200_431a_a592_c1bfe7cb53c6.slice/crio-7bca247292f025e446760401c0e2a5133df99bb03adb2d895f4c04f33b389231 WatchSource:0}: Error finding container 7bca247292f025e446760401c0e2a5133df99bb03adb2d895f4c04f33b389231: Status 404 returned error can't find the container with id 7bca247292f025e446760401c0e2a5133df99bb03adb2d895f4c04f33b389231 Jul 13 19:58:04.691289 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:04.691192 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j" event={"ID":"40814be3-8200-431a-a592-c1bfe7cb53c6","Type":"ContainerStarted","Data":"7bca247292f025e446760401c0e2a5133df99bb03adb2d895f4c04f33b389231"} Jul 13 19:58:07.710583 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:07.710477 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j" event={"ID":"40814be3-8200-431a-a592-c1bfe7cb53c6","Type":"ContainerStarted","Data":"b9034604205d7ac80b5c35880947a25e2f0afa4f41c0eab758244228bb5e5071"} Jul 13 19:58:07.726070 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:07.726012 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j" podStartSLOduration=2.118576273 podStartE2EDuration="4.725994206s" totalImagesPullingTime="2.607417933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:58:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:58:07.72439001 +0000 UTC m=+2182.838297718" watchObservedRunningTime="2026-07-13 19:58:07.725994206 +0000 UTC m=+2182.839901914" Jul 13 19:58:08.718711 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:08.718677 2556 generic.go:356] "Generic (PLEG): container finished" podID="40814be3-8200-431a-a592-c1bfe7cb53c6" containerID="b9034604205d7ac80b5c35880947a25e2f0afa4f41c0eab758244228bb5e5071" exitCode=0 Jul 13 19:58:08.719137 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:08.718770 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j" event={"ID":"40814be3-8200-431a-a592-c1bfe7cb53c6","Type":"ContainerDied","Data":"b9034604205d7ac80b5c35880947a25e2f0afa4f41c0eab758244228bb5e5071"} Jul 13 19:58:09.870073 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:09.870045 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j" Jul 13 19:58:09.901782 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:09.901729 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40814be3-8200-431a-a592-c1bfe7cb53c6-maas-api-service-ca\") pod \"40814be3-8200-431a-a592-c1bfe7cb53c6\" (UID: \"40814be3-8200-431a-a592-c1bfe7cb53c6\") " Jul 13 19:58:09.902112 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:09.902089 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40814be3-8200-431a-a592-c1bfe7cb53c6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "40814be3-8200-431a-a592-c1bfe7cb53c6" (UID: "40814be3-8200-431a-a592-c1bfe7cb53c6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:58:10.003558 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:10.003465 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40814be3-8200-431a-a592-c1bfe7cb53c6-maas-api-service-ca\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:58:10.732129 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:10.732095 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j" Jul 13 19:58:10.732129 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:10.732122 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j" event={"ID":"40814be3-8200-431a-a592-c1bfe7cb53c6","Type":"ContainerDied","Data":"7bca247292f025e446760401c0e2a5133df99bb03adb2d895f4c04f33b389231"} Jul 13 19:58:10.732349 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:10.732172 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7bca247292f025e446760401c0e2a5133df99bb03adb2d895f4c04f33b389231" Jul 13 19:58:14.448089 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.448050 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt"] Jul 13 19:58:14.448598 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.448302 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" podUID="9ced9247-d576-4280-b36e-4af381ff6f3d" containerName="maas-api" containerID="cri-o://d03a9aafe78b50a99a3fda1f570041296761beb266d8256d0046449932df6dd4" gracePeriod=30 Jul 13 19:58:14.714668 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.714597 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" Jul 13 19:58:14.750196 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.750144 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ced9247-d576-4280-b36e-4af381ff6f3d-maas-api-tls\") pod \"9ced9247-d576-4280-b36e-4af381ff6f3d\" (UID: \"9ced9247-d576-4280-b36e-4af381ff6f3d\") " Jul 13 19:58:14.750391 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.750348 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ced9247-d576-4280-b36e-4af381ff6f3d-openshift-service-ca\") pod \"9ced9247-d576-4280-b36e-4af381ff6f3d\" (UID: \"9ced9247-d576-4280-b36e-4af381ff6f3d\") " Jul 13 19:58:14.750472 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.750416 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4bm5\" (UniqueName: \"kubernetes.io/projected/9ced9247-d576-4280-b36e-4af381ff6f3d-kube-api-access-l4bm5\") pod \"9ced9247-d576-4280-b36e-4af381ff6f3d\" (UID: \"9ced9247-d576-4280-b36e-4af381ff6f3d\") " Jul 13 19:58:14.751002 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.750949 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ced9247-d576-4280-b36e-4af381ff6f3d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9ced9247-d576-4280-b36e-4af381ff6f3d" (UID: "9ced9247-d576-4280-b36e-4af381ff6f3d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:58:14.753144 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.753105 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ced9247-d576-4280-b36e-4af381ff6f3d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9ced9247-d576-4280-b36e-4af381ff6f3d" (UID: "9ced9247-d576-4280-b36e-4af381ff6f3d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:58:14.753280 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.753209 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ced9247-d576-4280-b36e-4af381ff6f3d-kube-api-access-l4bm5" (OuterVolumeSpecName: "kube-api-access-l4bm5") pod "9ced9247-d576-4280-b36e-4af381ff6f3d" (UID: "9ced9247-d576-4280-b36e-4af381ff6f3d"). InnerVolumeSpecName "kube-api-access-l4bm5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:58:14.759104 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.759069 2556 generic.go:356] "Generic (PLEG): container finished" podID="9ced9247-d576-4280-b36e-4af381ff6f3d" containerID="d03a9aafe78b50a99a3fda1f570041296761beb266d8256d0046449932df6dd4" exitCode=0 Jul 13 19:58:14.759275 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.759119 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" Jul 13 19:58:14.759275 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.759143 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" event={"ID":"9ced9247-d576-4280-b36e-4af381ff6f3d","Type":"ContainerDied","Data":"d03a9aafe78b50a99a3fda1f570041296761beb266d8256d0046449932df6dd4"} Jul 13 19:58:14.759275 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.759200 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt" event={"ID":"9ced9247-d576-4280-b36e-4af381ff6f3d","Type":"ContainerDied","Data":"43d711a89a6871ced4b9dcac85c9a95588ed7e8453c85fd81d8b2daee55365d5"} Jul 13 19:58:14.759275 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.759221 2556 scope.go:117] "RemoveContainer" containerID="d03a9aafe78b50a99a3fda1f570041296761beb266d8256d0046449932df6dd4" Jul 13 19:58:14.778866 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.778835 2556 scope.go:117] "RemoveContainer" containerID="d03a9aafe78b50a99a3fda1f570041296761beb266d8256d0046449932df6dd4" Jul 13 19:58:14.779216 ip-10-0-132-20 kubenswrapper[2556]: E0713 19:58:14.779195 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d03a9aafe78b50a99a3fda1f570041296761beb266d8256d0046449932df6dd4\": container with ID starting with d03a9aafe78b50a99a3fda1f570041296761beb266d8256d0046449932df6dd4 not found: ID does not exist" containerID="d03a9aafe78b50a99a3fda1f570041296761beb266d8256d0046449932df6dd4" Jul 13 19:58:14.779273 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.779228 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d03a9aafe78b50a99a3fda1f570041296761beb266d8256d0046449932df6dd4"} err="failed to get container status \"d03a9aafe78b50a99a3fda1f570041296761beb266d8256d0046449932df6dd4\": rpc error: code = NotFound desc = could not find container \"d03a9aafe78b50a99a3fda1f570041296761beb266d8256d0046449932df6dd4\": container with ID starting with d03a9aafe78b50a99a3fda1f570041296761beb266d8256d0046449932df6dd4 not found: ID does not exist" Jul 13 19:58:14.788175 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.788137 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt"] Jul 13 19:58:14.792471 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.792431 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d67d76ca-7888f4b9b5-cxjvt"] Jul 13 19:58:14.849920 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.849881 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ced9247-d576-4280-b36e-4af381ff6f3d" path="/var/lib/kubelet/pods/9ced9247-d576-4280-b36e-4af381ff6f3d/volumes" Jul 13 19:58:14.852165 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.852135 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ced9247-d576-4280-b36e-4af381ff6f3d-openshift-service-ca\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:58:14.852319 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.852176 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4bm5\" (UniqueName: \"kubernetes.io/projected/9ced9247-d576-4280-b36e-4af381ff6f3d-kube-api-access-l4bm5\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:58:14.852319 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:14.852188 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ced9247-d576-4280-b36e-4af381ff6f3d-maas-api-tls\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 19:58:16.698881 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:58:16.698835 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:59:26.105229 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:59:26.105186 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 19:59:40.211270 ip-10-0-132-20 kubenswrapper[2556]: I0713 19:59:40.211230 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:00:25.791714 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.791676 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5"] Jul 13 20:00:25.792182 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.792161 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40814be3-8200-431a-a592-c1bfe7cb53c6" containerName="revoke-keys" Jul 13 20:00:25.792182 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.792176 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="40814be3-8200-431a-a592-c1bfe7cb53c6" containerName="revoke-keys" Jul 13 20:00:25.792256 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.792193 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ced9247-d576-4280-b36e-4af381ff6f3d" containerName="maas-api" Jul 13 20:00:25.792256 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.792199 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ced9247-d576-4280-b36e-4af381ff6f3d" containerName="maas-api" Jul 13 20:00:25.792324 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.792258 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="40814be3-8200-431a-a592-c1bfe7cb53c6" containerName="revoke-keys" Jul 13 20:00:25.792324 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.792269 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ced9247-d576-4280-b36e-4af381ff6f3d" containerName="maas-api" Jul 13 20:00:25.795280 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.795259 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.797758 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.797734 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-7b8c26-openshift-default-dockercfg-gsxr8\"" Jul 13 20:00:25.808536 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.808499 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5"] Jul 13 20:00:25.894715 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.894660 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-workload-socket\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.894951 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.894791 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-podinfo\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.894951 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.894865 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-envoy\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.894951 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.894899 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a22b4c1-27ac-477a-86da-b868bea26c82-istiod-ca-cert\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.894951 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.894935 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-data\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.895249 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.895016 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-credential-socket\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.895249 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.895046 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-workload-certs\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.895249 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.895074 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-token\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.895249 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.895187 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56w8q\" (UniqueName: \"kubernetes.io/projected/3a22b4c1-27ac-477a-86da-b868bea26c82-kube-api-access-56w8q\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.996226 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.996180 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-workload-socket\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.996226 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.996230 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-podinfo\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.996512 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.996253 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-envoy\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.996512 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.996276 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a22b4c1-27ac-477a-86da-b868bea26c82-istiod-ca-cert\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.996512 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.996298 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-data\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.996512 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.996335 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-credential-socket\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.996512 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.996361 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-workload-certs\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.996810 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.996501 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-token\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.996810 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.996598 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56w8q\" (UniqueName: \"kubernetes.io/projected/3a22b4c1-27ac-477a-86da-b868bea26c82-kube-api-access-56w8q\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.996810 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.996755 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-workload-certs\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.996977 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.996831 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-data\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.997033 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.996998 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-workload-socket\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.997121 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.997100 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-credential-socket\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.997339 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.997321 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a22b4c1-27ac-477a-86da-b868bea26c82-istiod-ca-cert\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.999269 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.999246 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-envoy\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:25.999457 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:25.999440 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-podinfo\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:26.004957 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:26.004926 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-token\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:26.005122 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:26.005097 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56w8q\" (UniqueName: \"kubernetes.io/projected/3a22b4c1-27ac-477a-86da-b868bea26c82-kube-api-access-56w8q\") pod \"e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:26.107167 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:26.107067 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:26.262026 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:26.261992 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5"] Jul 13 20:00:26.265330 ip-10-0-132-20 kubenswrapper[2556]: W0713 20:00:26.265294 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a22b4c1_27ac_477a_86da_b868bea26c82.slice/crio-19d784e3c129877209be421697f607b09e54286226ac48244d4e1c6d3e3e9a12 WatchSource:0}: Error finding container 19d784e3c129877209be421697f607b09e54286226ac48244d4e1c6d3e3e9a12: Status 404 returned error can't find the container with id 19d784e3c129877209be421697f607b09e54286226ac48244d4e1c6d3e3e9a12 Jul 13 20:00:26.267838 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:26.267794 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 20:00:26.267935 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:26.267877 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 20:00:26.267935 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:26.267909 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 20:00:26.555715 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:26.555674 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" event={"ID":"3a22b4c1-27ac-477a-86da-b868bea26c82","Type":"ContainerStarted","Data":"c9239bcd505258ab509aeec24d63b6d6427c146a33081124ed19f2f29abc269c"} Jul 13 20:00:26.555961 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:26.555743 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" event={"ID":"3a22b4c1-27ac-477a-86da-b868bea26c82","Type":"ContainerStarted","Data":"19d784e3c129877209be421697f607b09e54286226ac48244d4e1c6d3e3e9a12"} Jul 13 20:00:26.594005 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:26.593928 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" podStartSLOduration=1.593903771 podStartE2EDuration="1.593903771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:00:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:00:26.592097533 +0000 UTC m=+2321.706005258" watchObservedRunningTime="2026-07-13 20:00:26.593903771 +0000 UTC m=+2321.707811481" Jul 13 20:00:27.107668 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:27.107633 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:27.113225 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:27.113198 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:27.562051 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:27.562016 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:27.563163 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:27.563136 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:29.211941 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:29.211898 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:00:43.603993 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:43.603937 2556 patch_prober.go:28] interesting pod/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 13 20:00:43.604524 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:43.604008 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" podUID="3a22b4c1-27ac-477a-86da-b868bea26c82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 13 20:00:44.107767 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:44.107715 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:00:44.671588 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:44.671441 2556 generic.go:356] "Generic (PLEG): container finished" podID="3a22b4c1-27ac-477a-86da-b868bea26c82" containerID="c9239bcd505258ab509aeec24d63b6d6427c146a33081124ed19f2f29abc269c" exitCode=0 Jul 13 20:00:44.671588 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:44.671518 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" event={"ID":"3a22b4c1-27ac-477a-86da-b868bea26c82","Type":"ContainerDied","Data":"c9239bcd505258ab509aeec24d63b6d6427c146a33081124ed19f2f29abc269c"} Jul 13 20:00:44.672074 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:44.672041 2556 scope.go:117] "RemoveContainer" containerID="c9239bcd505258ab509aeec24d63b6d6427c146a33081124ed19f2f29abc269c" Jul 13 20:00:44.673214 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:44.673186 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 20:00:44.673274 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:44.673261 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 20:00:44.673312 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:44.673296 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 20:00:45.679031 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:45.678992 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" event={"ID":"3a22b4c1-27ac-477a-86da-b868bea26c82","Type":"ContainerStarted","Data":"3e1d3491662f0d92339b299e0e85556efce8abacd222d733a8480c7984906d1b"} Jul 13 20:00:46.107597 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:46.107550 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:46.107923 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:46.107899 2556 patch_prober.go:28] interesting pod/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 13 20:00:46.107988 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:46.107966 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" podUID="3a22b4c1-27ac-477a-86da-b868bea26c82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 13 20:00:47.107586 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:47.107539 2556 patch_prober.go:28] interesting pod/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 13 20:00:47.108142 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:47.107619 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" podUID="3a22b4c1-27ac-477a-86da-b868bea26c82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 13 20:00:48.108306 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:48.108263 2556 patch_prober.go:28] interesting pod/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 13 20:00:48.108758 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:48.108326 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" podUID="3a22b4c1-27ac-477a-86da-b868bea26c82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 13 20:00:49.107721 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:49.107678 2556 patch_prober.go:28] interesting pod/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 13 20:00:49.108058 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:49.108026 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" podUID="3a22b4c1-27ac-477a-86da-b868bea26c82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 13 20:00:50.108319 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:50.108273 2556 patch_prober.go:28] interesting pod/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 13 20:00:50.108802 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:50.108339 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" podUID="3a22b4c1-27ac-477a-86da-b868bea26c82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 13 20:00:51.111890 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:51.111856 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:51.112337 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:51.112155 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:00:51.112979 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:00:51.112957 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:01:00.203417 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:01:00.203372 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:01:20.907068 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:01:20.907025 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:01:39.199422 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:01:39.199381 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:01:46.103535 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:01:46.103499 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 20:01:46.113411 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:01:46.113381 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 20:01:51.396858 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:01:51.396812 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:02:12.603383 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:12.603276 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:02:24.765342 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:24.765294 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5"] Jul 13 20:02:24.765940 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:24.765559 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" podUID="3a22b4c1-27ac-477a-86da-b868bea26c82" containerName="istio-proxy" containerID="cri-o://3e1d3491662f0d92339b299e0e85556efce8abacd222d733a8480c7984906d1b" gracePeriod=30 Jul 13 20:02:30.031456 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.031380 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:02:30.184218 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.184171 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56w8q\" (UniqueName: \"kubernetes.io/projected/3a22b4c1-27ac-477a-86da-b868bea26c82-kube-api-access-56w8q\") pod \"3a22b4c1-27ac-477a-86da-b868bea26c82\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " Jul 13 20:02:30.184409 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.184274 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-workload-socket\") pod \"3a22b4c1-27ac-477a-86da-b868bea26c82\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " Jul 13 20:02:30.184409 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.184380 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-envoy\") pod \"3a22b4c1-27ac-477a-86da-b868bea26c82\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " Jul 13 20:02:30.184493 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.184434 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-podinfo\") pod \"3a22b4c1-27ac-477a-86da-b868bea26c82\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " Jul 13 20:02:30.184493 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.184452 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-workload-certs\") pod \"3a22b4c1-27ac-477a-86da-b868bea26c82\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " Jul 13 20:02:30.184493 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.184469 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a22b4c1-27ac-477a-86da-b868bea26c82-istiod-ca-cert\") pod \"3a22b4c1-27ac-477a-86da-b868bea26c82\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " Jul 13 20:02:30.184716 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.184512 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-credential-socket\") pod \"3a22b4c1-27ac-477a-86da-b868bea26c82\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " Jul 13 20:02:30.184716 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.184540 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-token\") pod \"3a22b4c1-27ac-477a-86da-b868bea26c82\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " Jul 13 20:02:30.184716 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.184550 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3a22b4c1-27ac-477a-86da-b868bea26c82" (UID: "3a22b4c1-27ac-477a-86da-b868bea26c82"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:02:30.184716 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.184587 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-data\") pod \"3a22b4c1-27ac-477a-86da-b868bea26c82\" (UID: \"3a22b4c1-27ac-477a-86da-b868bea26c82\") " Jul 13 20:02:30.184963 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.184831 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3a22b4c1-27ac-477a-86da-b868bea26c82" (UID: "3a22b4c1-27ac-477a-86da-b868bea26c82"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:02:30.184963 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.184844 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3a22b4c1-27ac-477a-86da-b868bea26c82" (UID: "3a22b4c1-27ac-477a-86da-b868bea26c82"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:02:30.185078 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.185042 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-workload-certs\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:02:30.185078 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.185058 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-credential-socket\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:02:30.185078 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.185074 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-workload-socket\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:02:30.185228 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.185177 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a22b4c1-27ac-477a-86da-b868bea26c82-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3a22b4c1-27ac-477a-86da-b868bea26c82" (UID: "3a22b4c1-27ac-477a-86da-b868bea26c82"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:02:30.185563 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.185542 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-data" (OuterVolumeSpecName: "istio-data") pod "3a22b4c1-27ac-477a-86da-b868bea26c82" (UID: "3a22b4c1-27ac-477a-86da-b868bea26c82"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:02:30.186650 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.186624 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3a22b4c1-27ac-477a-86da-b868bea26c82" (UID: "3a22b4c1-27ac-477a-86da-b868bea26c82"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:02:30.186966 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.186931 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-token" (OuterVolumeSpecName: "istio-token") pod "3a22b4c1-27ac-477a-86da-b868bea26c82" (UID: "3a22b4c1-27ac-477a-86da-b868bea26c82"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:02:30.187072 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.187013 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3a22b4c1-27ac-477a-86da-b868bea26c82" (UID: "3a22b4c1-27ac-477a-86da-b868bea26c82"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 20:02:30.187072 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.187040 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a22b4c1-27ac-477a-86da-b868bea26c82-kube-api-access-56w8q" (OuterVolumeSpecName: "kube-api-access-56w8q") pod "3a22b4c1-27ac-477a-86da-b868bea26c82" (UID: "3a22b4c1-27ac-477a-86da-b868bea26c82"). InnerVolumeSpecName "kube-api-access-56w8q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:02:30.285932 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.285900 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-envoy\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:02:30.285932 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.285932 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-podinfo\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:02:30.286174 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.285943 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a22b4c1-27ac-477a-86da-b868bea26c82-istiod-ca-cert\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:02:30.286174 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.285951 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-token\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:02:30.286174 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.285959 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a22b4c1-27ac-477a-86da-b868bea26c82-istio-data\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:02:30.286174 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.285967 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56w8q\" (UniqueName: \"kubernetes.io/projected/3a22b4c1-27ac-477a-86da-b868bea26c82-kube-api-access-56w8q\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:02:30.322052 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.322017 2556 generic.go:356] "Generic (PLEG): container finished" podID="3a22b4c1-27ac-477a-86da-b868bea26c82" containerID="3e1d3491662f0d92339b299e0e85556efce8abacd222d733a8480c7984906d1b" exitCode=0 Jul 13 20:02:30.322235 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.322073 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" Jul 13 20:02:30.322235 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.322102 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" event={"ID":"3a22b4c1-27ac-477a-86da-b868bea26c82","Type":"ContainerDied","Data":"3e1d3491662f0d92339b299e0e85556efce8abacd222d733a8480c7984906d1b"} Jul 13 20:02:30.322235 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.322168 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5" event={"ID":"3a22b4c1-27ac-477a-86da-b868bea26c82","Type":"ContainerDied","Data":"19d784e3c129877209be421697f607b09e54286226ac48244d4e1c6d3e3e9a12"} Jul 13 20:02:30.322235 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.322195 2556 scope.go:117] "RemoveContainer" containerID="3e1d3491662f0d92339b299e0e85556efce8abacd222d733a8480c7984906d1b" Jul 13 20:02:30.333297 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.333274 2556 scope.go:117] "RemoveContainer" containerID="c9239bcd505258ab509aeec24d63b6d6427c146a33081124ed19f2f29abc269c" Jul 13 20:02:30.343505 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.343479 2556 scope.go:117] "RemoveContainer" containerID="3e1d3491662f0d92339b299e0e85556efce8abacd222d733a8480c7984906d1b" Jul 13 20:02:30.343896 ip-10-0-132-20 kubenswrapper[2556]: E0713 20:02:30.343872 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e1d3491662f0d92339b299e0e85556efce8abacd222d733a8480c7984906d1b\": container with ID starting with 3e1d3491662f0d92339b299e0e85556efce8abacd222d733a8480c7984906d1b not found: ID does not exist" containerID="3e1d3491662f0d92339b299e0e85556efce8abacd222d733a8480c7984906d1b" Jul 13 20:02:30.343958 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.343906 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e1d3491662f0d92339b299e0e85556efce8abacd222d733a8480c7984906d1b"} err="failed to get container status \"3e1d3491662f0d92339b299e0e85556efce8abacd222d733a8480c7984906d1b\": rpc error: code = NotFound desc = could not find container \"3e1d3491662f0d92339b299e0e85556efce8abacd222d733a8480c7984906d1b\": container with ID starting with 3e1d3491662f0d92339b299e0e85556efce8abacd222d733a8480c7984906d1b not found: ID does not exist" Jul 13 20:02:30.343958 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.343926 2556 scope.go:117] "RemoveContainer" containerID="c9239bcd505258ab509aeec24d63b6d6427c146a33081124ed19f2f29abc269c" Jul 13 20:02:30.344237 ip-10-0-132-20 kubenswrapper[2556]: E0713 20:02:30.344218 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9239bcd505258ab509aeec24d63b6d6427c146a33081124ed19f2f29abc269c\": container with ID starting with c9239bcd505258ab509aeec24d63b6d6427c146a33081124ed19f2f29abc269c not found: ID does not exist" containerID="c9239bcd505258ab509aeec24d63b6d6427c146a33081124ed19f2f29abc269c" Jul 13 20:02:30.344295 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.344242 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9239bcd505258ab509aeec24d63b6d6427c146a33081124ed19f2f29abc269c"} err="failed to get container status \"c9239bcd505258ab509aeec24d63b6d6427c146a33081124ed19f2f29abc269c\": rpc error: code = NotFound desc = could not find container \"c9239bcd505258ab509aeec24d63b6d6427c146a33081124ed19f2f29abc269c\": container with ID starting with c9239bcd505258ab509aeec24d63b6d6427c146a33081124ed19f2f29abc269c not found: ID does not exist" Jul 13 20:02:30.344990 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.344968 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5"] Jul 13 20:02:30.349578 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.349545 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-7b8c26-openshift-default-87c58df9-q98p5"] Jul 13 20:02:30.850319 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:30.850281 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a22b4c1-27ac-477a-86da-b868bea26c82" path="/var/lib/kubelet/pods/3a22b4c1-27ac-477a-86da-b868bea26c82/volumes" Jul 13 20:02:35.403608 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:35.403566 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:02:36.407434 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.407399 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9"] Jul 13 20:02:36.408046 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.408024 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a22b4c1-27ac-477a-86da-b868bea26c82" containerName="istio-proxy" Jul 13 20:02:36.408131 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.408048 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a22b4c1-27ac-477a-86da-b868bea26c82" containerName="istio-proxy" Jul 13 20:02:36.408131 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.408102 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a22b4c1-27ac-477a-86da-b868bea26c82" containerName="istio-proxy" Jul 13 20:02:36.408131 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.408112 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a22b4c1-27ac-477a-86da-b868bea26c82" containerName="istio-proxy" Jul 13 20:02:36.408282 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.408202 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a22b4c1-27ac-477a-86da-b868bea26c82" containerName="istio-proxy" Jul 13 20:02:36.411623 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.411597 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" Jul 13 20:02:36.414462 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.414412 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-f221d9-serving-cert\"" Jul 13 20:02:36.414626 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.414414 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 20:02:36.414713 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.414675 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 20:02:36.414803 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.414763 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q6tkl\"" Jul 13 20:02:36.423035 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.422901 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9"] Jul 13 20:02:36.446334 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.446289 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n4bd\" (UniqueName: \"kubernetes.io/projected/77494e95-1a27-4fb0-86d8-505d415c69e1-kube-api-access-4n4bd\") pod \"maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9\" (UID: \"77494e95-1a27-4fb0-86d8-505d415c69e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" Jul 13 20:02:36.446506 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.446349 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77494e95-1a27-4fb0-86d8-505d415c69e1-openshift-service-ca\") pod \"maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9\" (UID: \"77494e95-1a27-4fb0-86d8-505d415c69e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" Jul 13 20:02:36.446506 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.446396 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77494e95-1a27-4fb0-86d8-505d415c69e1-maas-api-tls\") pod \"maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9\" (UID: \"77494e95-1a27-4fb0-86d8-505d415c69e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" Jul 13 20:02:36.547145 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.547096 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n4bd\" (UniqueName: \"kubernetes.io/projected/77494e95-1a27-4fb0-86d8-505d415c69e1-kube-api-access-4n4bd\") pod \"maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9\" (UID: \"77494e95-1a27-4fb0-86d8-505d415c69e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" Jul 13 20:02:36.547145 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.547145 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77494e95-1a27-4fb0-86d8-505d415c69e1-openshift-service-ca\") pod \"maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9\" (UID: \"77494e95-1a27-4fb0-86d8-505d415c69e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" Jul 13 20:02:36.547418 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.547182 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77494e95-1a27-4fb0-86d8-505d415c69e1-maas-api-tls\") pod \"maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9\" (UID: \"77494e95-1a27-4fb0-86d8-505d415c69e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" Jul 13 20:02:36.547418 ip-10-0-132-20 kubenswrapper[2556]: E0713 20:02:36.547296 2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-serving-cert: secret "maas-api-e2e-api-b-f221d9-serving-cert" not found Jul 13 20:02:36.547418 ip-10-0-132-20 kubenswrapper[2556]: E0713 20:02:36.547372 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77494e95-1a27-4fb0-86d8-505d415c69e1-maas-api-tls podName:77494e95-1a27-4fb0-86d8-505d415c69e1 nodeName:}" failed. No retries permitted until 2026-07-13 20:02:37.047340836 +0000 UTC m=+2452.161248522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/77494e95-1a27-4fb0-86d8-505d415c69e1-maas-api-tls") pod "maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" (UID: "77494e95-1a27-4fb0-86d8-505d415c69e1") : secret "maas-api-e2e-api-b-f221d9-serving-cert" not found Jul 13 20:02:36.548093 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.548060 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77494e95-1a27-4fb0-86d8-505d415c69e1-openshift-service-ca\") pod \"maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9\" (UID: \"77494e95-1a27-4fb0-86d8-505d415c69e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" Jul 13 20:02:36.562116 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:36.562080 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n4bd\" (UniqueName: \"kubernetes.io/projected/77494e95-1a27-4fb0-86d8-505d415c69e1-kube-api-access-4n4bd\") pod \"maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9\" (UID: \"77494e95-1a27-4fb0-86d8-505d415c69e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" Jul 13 20:02:37.051391 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:37.051346 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77494e95-1a27-4fb0-86d8-505d415c69e1-maas-api-tls\") pod \"maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9\" (UID: \"77494e95-1a27-4fb0-86d8-505d415c69e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" Jul 13 20:02:37.053998 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:37.053977 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77494e95-1a27-4fb0-86d8-505d415c69e1-maas-api-tls\") pod \"maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9\" (UID: \"77494e95-1a27-4fb0-86d8-505d415c69e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" Jul 13 20:02:37.326458 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:37.326352 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" Jul 13 20:02:37.480433 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:37.480397 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9"] Jul 13 20:02:37.480770 ip-10-0-132-20 kubenswrapper[2556]: W0713 20:02:37.480536 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77494e95_1a27_4fb0_86d8_505d415c69e1.slice/crio-7b5b3d9f2232b1d8e70baecf71cf0165ff42ef7adb0288025bf8ef8952ed84d5 WatchSource:0}: Error finding container 7b5b3d9f2232b1d8e70baecf71cf0165ff42ef7adb0288025bf8ef8952ed84d5: Status 404 returned error can't find the container with id 7b5b3d9f2232b1d8e70baecf71cf0165ff42ef7adb0288025bf8ef8952ed84d5 Jul 13 20:02:37.481998 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:37.481979 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 20:02:38.375171 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:38.375127 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" event={"ID":"77494e95-1a27-4fb0-86d8-505d415c69e1","Type":"ContainerStarted","Data":"7b5b3d9f2232b1d8e70baecf71cf0165ff42ef7adb0288025bf8ef8952ed84d5"} Jul 13 20:02:40.389929 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:40.389884 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" event={"ID":"77494e95-1a27-4fb0-86d8-505d415c69e1","Type":"ContainerStarted","Data":"5d00f29efc4cc23ff7fc80e72a0ba6e315f9515979798560c7fe3af4fecbad1d"} Jul 13 20:02:40.390450 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:40.390113 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" Jul 13 20:02:40.407248 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:40.407186 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" podStartSLOduration=1.885818233 podStartE2EDuration="4.407165685s" totalImagesPullingTime="2.521347452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:02:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:02:40.40566049 +0000 UTC m=+2455.519568233" watchObservedRunningTime="2026-07-13 20:02:40.407165685 +0000 UTC m=+2455.521073394" Jul 13 20:02:46.401670 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:46.401632 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" Jul 13 20:02:52.297311 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:02:52.297264 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:03:01.804261 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:01.804215 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:03:25.482697 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:25.482655 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9"] Jul 13 20:03:25.483265 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:25.482940 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" podUID="77494e95-1a27-4fb0-86d8-505d415c69e1" containerName="maas-api" containerID="cri-o://5d00f29efc4cc23ff7fc80e72a0ba6e315f9515979798560c7fe3af4fecbad1d" gracePeriod=30 Jul 13 20:03:25.660006 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:25.659970 2556 generic.go:356] "Generic (PLEG): container finished" podID="77494e95-1a27-4fb0-86d8-505d415c69e1" containerID="5d00f29efc4cc23ff7fc80e72a0ba6e315f9515979798560c7fe3af4fecbad1d" exitCode=0 Jul 13 20:03:25.660181 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:25.660154 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" event={"ID":"77494e95-1a27-4fb0-86d8-505d415c69e1","Type":"ContainerDied","Data":"5d00f29efc4cc23ff7fc80e72a0ba6e315f9515979798560c7fe3af4fecbad1d"} Jul 13 20:03:25.741411 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:25.741336 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" Jul 13 20:03:25.854161 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:25.854125 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4n4bd\" (UniqueName: \"kubernetes.io/projected/77494e95-1a27-4fb0-86d8-505d415c69e1-kube-api-access-4n4bd\") pod \"77494e95-1a27-4fb0-86d8-505d415c69e1\" (UID: \"77494e95-1a27-4fb0-86d8-505d415c69e1\") " Jul 13 20:03:25.854364 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:25.854194 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77494e95-1a27-4fb0-86d8-505d415c69e1-openshift-service-ca\") pod \"77494e95-1a27-4fb0-86d8-505d415c69e1\" (UID: \"77494e95-1a27-4fb0-86d8-505d415c69e1\") " Jul 13 20:03:25.854364 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:25.854217 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77494e95-1a27-4fb0-86d8-505d415c69e1-maas-api-tls\") pod \"77494e95-1a27-4fb0-86d8-505d415c69e1\" (UID: \"77494e95-1a27-4fb0-86d8-505d415c69e1\") " Jul 13 20:03:25.854575 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:25.854544 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77494e95-1a27-4fb0-86d8-505d415c69e1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "77494e95-1a27-4fb0-86d8-505d415c69e1" (UID: "77494e95-1a27-4fb0-86d8-505d415c69e1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:03:25.856429 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:25.856401 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77494e95-1a27-4fb0-86d8-505d415c69e1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "77494e95-1a27-4fb0-86d8-505d415c69e1" (UID: "77494e95-1a27-4fb0-86d8-505d415c69e1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:03:25.856552 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:25.856460 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77494e95-1a27-4fb0-86d8-505d415c69e1-kube-api-access-4n4bd" (OuterVolumeSpecName: "kube-api-access-4n4bd") pod "77494e95-1a27-4fb0-86d8-505d415c69e1" (UID: "77494e95-1a27-4fb0-86d8-505d415c69e1"). InnerVolumeSpecName "kube-api-access-4n4bd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:03:25.955769 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:25.955727 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4n4bd\" (UniqueName: \"kubernetes.io/projected/77494e95-1a27-4fb0-86d8-505d415c69e1-kube-api-access-4n4bd\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:03:25.955769 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:25.955760 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77494e95-1a27-4fb0-86d8-505d415c69e1-openshift-service-ca\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:03:25.955769 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:25.955784 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77494e95-1a27-4fb0-86d8-505d415c69e1-maas-api-tls\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:03:26.667332 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:26.667288 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" event={"ID":"77494e95-1a27-4fb0-86d8-505d415c69e1","Type":"ContainerDied","Data":"7b5b3d9f2232b1d8e70baecf71cf0165ff42ef7adb0288025bf8ef8952ed84d5"} Jul 13 20:03:26.667994 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:26.667356 2556 scope.go:117] "RemoveContainer" containerID="5d00f29efc4cc23ff7fc80e72a0ba6e315f9515979798560c7fe3af4fecbad1d" Jul 13 20:03:26.667994 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:26.667313 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9" Jul 13 20:03:26.694207 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:26.694162 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9"] Jul 13 20:03:26.698404 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:26.698375 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f221d9-59466f8f99-5dzs9"] Jul 13 20:03:26.850165 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:26.850131 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77494e95-1a27-4fb0-86d8-505d415c69e1" path="/var/lib/kubelet/pods/77494e95-1a27-4fb0-86d8-505d415c69e1/volumes" Jul 13 20:03:29.396454 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:29.396408 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:03:44.618503 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.618390 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds"] Jul 13 20:03:44.619011 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.618888 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77494e95-1a27-4fb0-86d8-505d415c69e1" containerName="maas-api" Jul 13 20:03:44.619011 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.618902 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="77494e95-1a27-4fb0-86d8-505d415c69e1" containerName="maas-api" Jul 13 20:03:44.619011 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.619008 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a22b4c1-27ac-477a-86da-b868bea26c82" containerName="istio-proxy" Jul 13 20:03:44.619172 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.619026 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="77494e95-1a27-4fb0-86d8-505d415c69e1" containerName="maas-api" Jul 13 20:03:44.623922 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.623893 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" Jul 13 20:03:44.626010 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.625978 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-452240-serving-cert\"" Jul 13 20:03:44.626231 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.626158 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 20:03:44.626909 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.626883 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 20:03:44.627071 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.626894 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q6tkl\"" Jul 13 20:03:44.632882 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.632797 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds"] Jul 13 20:03:44.645727 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.645694 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29cx2\" (UniqueName: \"kubernetes.io/projected/1ebed944-88fa-4bbc-ad43-8dbf532e972e-kube-api-access-29cx2\") pod \"maas-api-e2e-shared-a-452240-5dc69587b8-ttbds\" (UID: \"1ebed944-88fa-4bbc-ad43-8dbf532e972e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" Jul 13 20:03:44.645924 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.645763 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ebed944-88fa-4bbc-ad43-8dbf532e972e-maas-api-tls\") pod \"maas-api-e2e-shared-a-452240-5dc69587b8-ttbds\" (UID: \"1ebed944-88fa-4bbc-ad43-8dbf532e972e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" Jul 13 20:03:44.645924 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.645820 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ebed944-88fa-4bbc-ad43-8dbf532e972e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-452240-5dc69587b8-ttbds\" (UID: \"1ebed944-88fa-4bbc-ad43-8dbf532e972e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" Jul 13 20:03:44.746330 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.746291 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ebed944-88fa-4bbc-ad43-8dbf532e972e-maas-api-tls\") pod \"maas-api-e2e-shared-a-452240-5dc69587b8-ttbds\" (UID: \"1ebed944-88fa-4bbc-ad43-8dbf532e972e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" Jul 13 20:03:44.746541 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.746340 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ebed944-88fa-4bbc-ad43-8dbf532e972e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-452240-5dc69587b8-ttbds\" (UID: \"1ebed944-88fa-4bbc-ad43-8dbf532e972e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" Jul 13 20:03:44.746541 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.746446 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29cx2\" (UniqueName: \"kubernetes.io/projected/1ebed944-88fa-4bbc-ad43-8dbf532e972e-kube-api-access-29cx2\") pod \"maas-api-e2e-shared-a-452240-5dc69587b8-ttbds\" (UID: \"1ebed944-88fa-4bbc-ad43-8dbf532e972e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" Jul 13 20:03:44.746541 ip-10-0-132-20 kubenswrapper[2556]: E0713 20:03:44.746465 2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-serving-cert: secret "maas-api-e2e-shared-a-452240-serving-cert" not found Jul 13 20:03:44.746716 ip-10-0-132-20 kubenswrapper[2556]: E0713 20:03:44.746546 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ebed944-88fa-4bbc-ad43-8dbf532e972e-maas-api-tls podName:1ebed944-88fa-4bbc-ad43-8dbf532e972e nodeName:}" failed. No retries permitted until 2026-07-13 20:03:45.246522417 +0000 UTC m=+2520.360430108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1ebed944-88fa-4bbc-ad43-8dbf532e972e-maas-api-tls") pod "maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" (UID: "1ebed944-88fa-4bbc-ad43-8dbf532e972e") : secret "maas-api-e2e-shared-a-452240-serving-cert" not found Jul 13 20:03:44.747282 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.747259 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ebed944-88fa-4bbc-ad43-8dbf532e972e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-452240-5dc69587b8-ttbds\" (UID: \"1ebed944-88fa-4bbc-ad43-8dbf532e972e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" Jul 13 20:03:44.756586 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:44.756548 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29cx2\" (UniqueName: \"kubernetes.io/projected/1ebed944-88fa-4bbc-ad43-8dbf532e972e-kube-api-access-29cx2\") pod \"maas-api-e2e-shared-a-452240-5dc69587b8-ttbds\" (UID: \"1ebed944-88fa-4bbc-ad43-8dbf532e972e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" Jul 13 20:03:45.250427 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:45.250380 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ebed944-88fa-4bbc-ad43-8dbf532e972e-maas-api-tls\") pod \"maas-api-e2e-shared-a-452240-5dc69587b8-ttbds\" (UID: \"1ebed944-88fa-4bbc-ad43-8dbf532e972e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" Jul 13 20:03:45.253005 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:45.252980 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ebed944-88fa-4bbc-ad43-8dbf532e972e-maas-api-tls\") pod \"maas-api-e2e-shared-a-452240-5dc69587b8-ttbds\" (UID: \"1ebed944-88fa-4bbc-ad43-8dbf532e972e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" Jul 13 20:03:45.540585 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:45.540539 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" Jul 13 20:03:45.696506 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:45.696473 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds"] Jul 13 20:03:45.698406 ip-10-0-132-20 kubenswrapper[2556]: W0713 20:03:45.698379 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ebed944_88fa_4bbc_ad43_8dbf532e972e.slice/crio-fb0678509b467214cd39be738f4b1a938b655bd66f341fd06e83f841c7b9826c WatchSource:0}: Error finding container fb0678509b467214cd39be738f4b1a938b655bd66f341fd06e83f841c7b9826c: Status 404 returned error can't find the container with id fb0678509b467214cd39be738f4b1a938b655bd66f341fd06e83f841c7b9826c Jul 13 20:03:45.782422 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:45.782374 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" event={"ID":"1ebed944-88fa-4bbc-ad43-8dbf532e972e","Type":"ContainerStarted","Data":"fb0678509b467214cd39be738f4b1a938b655bd66f341fd06e83f841c7b9826c"} Jul 13 20:03:46.790542 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:46.790499 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" event={"ID":"1ebed944-88fa-4bbc-ad43-8dbf532e972e","Type":"ContainerStarted","Data":"21ef26b58efe1e4544a60c2f7ca3cc81e4d0003fca61b1f9d2447b65dab45400"} Jul 13 20:03:46.791011 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:46.790605 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" Jul 13 20:03:46.809372 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:46.809318 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" podStartSLOduration=1.853463535 podStartE2EDuration="2.809299751s" totalImagesPullingTime="955.836216ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:03:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:03:46.805898161 +0000 UTC m=+2521.919805869" watchObservedRunningTime="2026-07-13 20:03:46.809299751 +0000 UTC m=+2521.923207487" Jul 13 20:03:50.509173 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:50.509129 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:03:52.804482 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:52.804446 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" Jul 13 20:03:52.895893 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:52.895853 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q"] Jul 13 20:03:52.899702 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:52.899674 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" Jul 13 20:03:52.901799 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:52.901758 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-83eaac-serving-cert\"" Jul 13 20:03:52.911299 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:52.911228 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q"] Jul 13 20:03:52.928836 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:52.927443 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmp5b\" (UniqueName: \"kubernetes.io/projected/5d3e1560-2993-4412-a65f-7092a067ed2e-kube-api-access-kmp5b\") pod \"maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q\" (UID: \"5d3e1560-2993-4412-a65f-7092a067ed2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" Jul 13 20:03:52.928836 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:52.927610 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d3e1560-2993-4412-a65f-7092a067ed2e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q\" (UID: \"5d3e1560-2993-4412-a65f-7092a067ed2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" Jul 13 20:03:52.928836 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:52.927738 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d3e1560-2993-4412-a65f-7092a067ed2e-maas-api-tls\") pod \"maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q\" (UID: \"5d3e1560-2993-4412-a65f-7092a067ed2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" Jul 13 20:03:53.028382 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:53.028335 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmp5b\" (UniqueName: \"kubernetes.io/projected/5d3e1560-2993-4412-a65f-7092a067ed2e-kube-api-access-kmp5b\") pod \"maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q\" (UID: \"5d3e1560-2993-4412-a65f-7092a067ed2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" Jul 13 20:03:53.028604 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:53.028428 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d3e1560-2993-4412-a65f-7092a067ed2e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q\" (UID: \"5d3e1560-2993-4412-a65f-7092a067ed2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" Jul 13 20:03:53.028604 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:53.028583 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d3e1560-2993-4412-a65f-7092a067ed2e-maas-api-tls\") pod \"maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q\" (UID: \"5d3e1560-2993-4412-a65f-7092a067ed2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" Jul 13 20:03:53.028809 ip-10-0-132-20 kubenswrapper[2556]: E0713 20:03:53.028765 2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-serving-cert: secret "maas-api-e2e-shared-b-83eaac-serving-cert" not found Jul 13 20:03:53.028909 ip-10-0-132-20 kubenswrapper[2556]: E0713 20:03:53.028896 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d3e1560-2993-4412-a65f-7092a067ed2e-maas-api-tls podName:5d3e1560-2993-4412-a65f-7092a067ed2e nodeName:}" failed. No retries permitted until 2026-07-13 20:03:53.528868775 +0000 UTC m=+2528.642776474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5d3e1560-2993-4412-a65f-7092a067ed2e-maas-api-tls") pod "maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" (UID: "5d3e1560-2993-4412-a65f-7092a067ed2e") : secret "maas-api-e2e-shared-b-83eaac-serving-cert" not found Jul 13 20:03:53.029113 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:53.029096 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d3e1560-2993-4412-a65f-7092a067ed2e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q\" (UID: \"5d3e1560-2993-4412-a65f-7092a067ed2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" Jul 13 20:03:53.039608 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:53.039573 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmp5b\" (UniqueName: \"kubernetes.io/projected/5d3e1560-2993-4412-a65f-7092a067ed2e-kube-api-access-kmp5b\") pod \"maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q\" (UID: \"5d3e1560-2993-4412-a65f-7092a067ed2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" Jul 13 20:03:53.533560 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:53.533517 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d3e1560-2993-4412-a65f-7092a067ed2e-maas-api-tls\") pod \"maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q\" (UID: \"5d3e1560-2993-4412-a65f-7092a067ed2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" Jul 13 20:03:53.536344 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:53.536309 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d3e1560-2993-4412-a65f-7092a067ed2e-maas-api-tls\") pod \"maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q\" (UID: \"5d3e1560-2993-4412-a65f-7092a067ed2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" Jul 13 20:03:53.813153 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:53.813048 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" Jul 13 20:03:53.966938 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:53.966910 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q"] Jul 13 20:03:53.968907 ip-10-0-132-20 kubenswrapper[2556]: W0713 20:03:53.968874 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d3e1560_2993_4412_a65f_7092a067ed2e.slice/crio-510a36cbcb4dc47eb377cc8f46079bfc24a5a0a5d4be50bf1d0c7a5be1d5d57a WatchSource:0}: Error finding container 510a36cbcb4dc47eb377cc8f46079bfc24a5a0a5d4be50bf1d0c7a5be1d5d57a: Status 404 returned error can't find the container with id 510a36cbcb4dc47eb377cc8f46079bfc24a5a0a5d4be50bf1d0c7a5be1d5d57a Jul 13 20:03:54.849872 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:54.849843 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" Jul 13 20:03:54.850287 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:54.849896 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" event={"ID":"5d3e1560-2993-4412-a65f-7092a067ed2e","Type":"ContainerStarted","Data":"e20648baaf7c197736ef58a4810fca6849575e2ab87c7e5382084ea4b81b32d7"} Jul 13 20:03:54.850287 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:54.849915 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" event={"ID":"5d3e1560-2993-4412-a65f-7092a067ed2e","Type":"ContainerStarted","Data":"510a36cbcb4dc47eb377cc8f46079bfc24a5a0a5d4be50bf1d0c7a5be1d5d57a"} Jul 13 20:03:54.861466 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:03:54.861412 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" podStartSLOduration=2.4122806150000002 podStartE2EDuration="2.86139125s" totalImagesPullingTime="449.110635ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:03:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:03:54.8595889 +0000 UTC m=+2529.973496610" watchObservedRunningTime="2026-07-13 20:03:54.86139125 +0000 UTC m=+2529.975298960" Jul 13 20:04:00.856824 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:04:00.856789 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" Jul 13 20:04:03.710283 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:04:03.710243 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:04:12.805684 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:04:12.805645 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:04:47.793890 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:04:47.793844 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:05:07.499662 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:05:07.499572 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:05:26.811890 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:05:26.811844 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:05:44.705671 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:05:44.705630 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:06:01.400292 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:06:01.400247 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:06:21.510223 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:06:21.510180 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:06:37.199631 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:06:37.199543 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:06:46.145526 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:06:46.145491 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 20:06:46.156403 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:06:46.156373 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 20:06:57.101970 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:06:57.101926 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:07:17.541267 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:07:17.541225 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:07:44.496972 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:07:44.496932 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:08:00.606224 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:08:00.606185 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:08:21.306690 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:08:21.306599 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:08:41.400263 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:08:41.400227 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:08:55.094662 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:08:55.094618 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:09:21.299362 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:09:21.299316 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:09:36.801624 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:09:36.801527 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:10:00.418183 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:00.418145 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:10:06.241438 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:06.241394 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7702-139685475399856"] Jul 13 20:10:06.245447 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:06.245425 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475399856" Jul 13 20:10:06.247480 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:06.247458 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-rddtv\"" Jul 13 20:10:06.251478 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:06.251455 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7702-139685475399856"] Jul 13 20:10:06.290378 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:06.290327 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcc8s\" (UniqueName: \"kubernetes.io/projected/daa5968c-f07f-40ac-afdd-0553474ab178-kube-api-access-jcc8s\") pod \"test-curl-7702-139685475399856\" (UID: \"daa5968c-f07f-40ac-afdd-0553474ab178\") " pod="openshift-ingress/test-curl-7702-139685475399856" Jul 13 20:10:06.391909 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:06.391864 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcc8s\" (UniqueName: \"kubernetes.io/projected/daa5968c-f07f-40ac-afdd-0553474ab178-kube-api-access-jcc8s\") pod \"test-curl-7702-139685475399856\" (UID: \"daa5968c-f07f-40ac-afdd-0553474ab178\") " pod="openshift-ingress/test-curl-7702-139685475399856" Jul 13 20:10:06.401208 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:06.401178 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcc8s\" (UniqueName: \"kubernetes.io/projected/daa5968c-f07f-40ac-afdd-0553474ab178-kube-api-access-jcc8s\") pod \"test-curl-7702-139685475399856\" (UID: \"daa5968c-f07f-40ac-afdd-0553474ab178\") " pod="openshift-ingress/test-curl-7702-139685475399856" Jul 13 20:10:06.557453 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:06.557400 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475399856" Jul 13 20:10:06.710053 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:06.710022 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7702-139685475399856"] Jul 13 20:10:06.712712 ip-10-0-132-20 kubenswrapper[2556]: W0713 20:10:06.712679 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddaa5968c_f07f_40ac_afdd_0553474ab178.slice/crio-2c43f4e9139ac13f73a03409ea697b9bdeb5df4bfeee134dc400e9a4b48ca7db WatchSource:0}: Error finding container 2c43f4e9139ac13f73a03409ea697b9bdeb5df4bfeee134dc400e9a4b48ca7db: Status 404 returned error can't find the container with id 2c43f4e9139ac13f73a03409ea697b9bdeb5df4bfeee134dc400e9a4b48ca7db Jul 13 20:10:06.713946 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:06.713928 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 20:10:07.136628 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:07.136579 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7702-139685475399856" event={"ID":"daa5968c-f07f-40ac-afdd-0553474ab178","Type":"ContainerStarted","Data":"2c43f4e9139ac13f73a03409ea697b9bdeb5df4bfeee134dc400e9a4b48ca7db"} Jul 13 20:10:08.144970 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:08.144929 2556 generic.go:356] "Generic (PLEG): container finished" podID="daa5968c-f07f-40ac-afdd-0553474ab178" containerID="7186b12af8d0e63197a8cbf2240f72b4a9a3e73b9c43f3ed124f5946f18cd8c3" exitCode=0 Jul 13 20:10:08.145477 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:08.145019 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7702-139685475399856" event={"ID":"daa5968c-f07f-40ac-afdd-0553474ab178","Type":"ContainerDied","Data":"7186b12af8d0e63197a8cbf2240f72b4a9a3e73b9c43f3ed124f5946f18cd8c3"} Jul 13 20:10:09.305677 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:09.305645 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475399856" Jul 13 20:10:09.345878 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:09.345837 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7702-139685475399856_daa5968c-f07f-40ac-afdd-0553474ab178/test-curl-7702-139685475399856/0.log" Jul 13 20:10:09.370356 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:09.370316 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7702-139685475399856"] Jul 13 20:10:09.376950 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:09.376914 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7702-139685475399856"] Jul 13 20:10:09.424891 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:09.424756 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcc8s\" (UniqueName: \"kubernetes.io/projected/daa5968c-f07f-40ac-afdd-0553474ab178-kube-api-access-jcc8s\") pod \"daa5968c-f07f-40ac-afdd-0553474ab178\" (UID: \"daa5968c-f07f-40ac-afdd-0553474ab178\") " Jul 13 20:10:09.427080 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:09.427039 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daa5968c-f07f-40ac-afdd-0553474ab178-kube-api-access-jcc8s" (OuterVolumeSpecName: "kube-api-access-jcc8s") pod "daa5968c-f07f-40ac-afdd-0553474ab178" (UID: "daa5968c-f07f-40ac-afdd-0553474ab178"). InnerVolumeSpecName "kube-api-access-jcc8s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:10:09.505247 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:09.505205 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7702-139685475398704"] Jul 13 20:10:09.505673 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:09.505660 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daa5968c-f07f-40ac-afdd-0553474ab178" containerName="test-curl-7702-139685475399856" Jul 13 20:10:09.505724 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:09.505675 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="daa5968c-f07f-40ac-afdd-0553474ab178" containerName="test-curl-7702-139685475399856" Jul 13 20:10:09.505769 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:09.505764 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="daa5968c-f07f-40ac-afdd-0553474ab178" containerName="test-curl-7702-139685475399856" Jul 13 20:10:09.512134 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:09.510279 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475398704" Jul 13 20:10:09.519452 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:09.519421 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7702-139685475398704"] Jul 13 20:10:09.525761 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:09.525734 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcc8s\" (UniqueName: \"kubernetes.io/projected/daa5968c-f07f-40ac-afdd-0553474ab178-kube-api-access-jcc8s\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:10:09.627059 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:09.627020 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l759\" (UniqueName: \"kubernetes.io/projected/6677d4b5-764f-4be0-bf5c-c70e1fd185d5-kube-api-access-4l759\") pod \"test-curl-7702-139685475398704\" (UID: \"6677d4b5-764f-4be0-bf5c-c70e1fd185d5\") " pod="openshift-ingress/test-curl-7702-139685475398704" Jul 13 20:10:09.728629 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:09.728536 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l759\" (UniqueName: \"kubernetes.io/projected/6677d4b5-764f-4be0-bf5c-c70e1fd185d5-kube-api-access-4l759\") pod \"test-curl-7702-139685475398704\" (UID: \"6677d4b5-764f-4be0-bf5c-c70e1fd185d5\") " pod="openshift-ingress/test-curl-7702-139685475398704" Jul 13 20:10:09.736935 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:09.736900 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l759\" (UniqueName: \"kubernetes.io/projected/6677d4b5-764f-4be0-bf5c-c70e1fd185d5-kube-api-access-4l759\") pod \"test-curl-7702-139685475398704\" (UID: \"6677d4b5-764f-4be0-bf5c-c70e1fd185d5\") " pod="openshift-ingress/test-curl-7702-139685475398704" Jul 13 20:10:09.825749 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:09.825703 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475398704" Jul 13 20:10:09.977885 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:09.977843 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7702-139685475398704"] Jul 13 20:10:09.996757 ip-10-0-132-20 kubenswrapper[2556]: W0713 20:10:09.996720 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6677d4b5_764f_4be0_bf5c_c70e1fd185d5.slice/crio-fd7b2b11f9c192f21d75a03d6d25a7b0ea7cf98b7086cbd559938bb901c34f4b WatchSource:0}: Error finding container fd7b2b11f9c192f21d75a03d6d25a7b0ea7cf98b7086cbd559938bb901c34f4b: Status 404 returned error can't find the container with id fd7b2b11f9c192f21d75a03d6d25a7b0ea7cf98b7086cbd559938bb901c34f4b Jul 13 20:10:10.158430 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:10.158398 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475399856" Jul 13 20:10:10.158430 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:10.158423 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c43f4e9139ac13f73a03409ea697b9bdeb5df4bfeee134dc400e9a4b48ca7db" Jul 13 20:10:10.159726 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:10.159697 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7702-139685475398704" event={"ID":"6677d4b5-764f-4be0-bf5c-c70e1fd185d5","Type":"ContainerStarted","Data":"fd7b2b11f9c192f21d75a03d6d25a7b0ea7cf98b7086cbd559938bb901c34f4b"} Jul 13 20:10:10.169034 ip-10-0-132-20 kubenswrapper[2556]: E0713 20:10:10.169003 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7702-139685475399856\" is forbidden: User \"system:node:ip-10-0-132-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-132-20.ec2.internal' and this object" podUID="daa5968c-f07f-40ac-afdd-0553474ab178" pod="openshift-ingress/test-curl-7702-139685475399856" Jul 13 20:10:10.850138 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:10.850099 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daa5968c-f07f-40ac-afdd-0553474ab178" path="/var/lib/kubelet/pods/daa5968c-f07f-40ac-afdd-0553474ab178/volumes" Jul 13 20:10:11.167770 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:11.167677 2556 generic.go:356] "Generic (PLEG): container finished" podID="6677d4b5-764f-4be0-bf5c-c70e1fd185d5" containerID="dffc05feeecacd8fb59242cef52027e30813e6e0daeafd2db6f17bc5381db69c" exitCode=0 Jul 13 20:10:11.167932 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:11.167766 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7702-139685475398704" event={"ID":"6677d4b5-764f-4be0-bf5c-c70e1fd185d5","Type":"ContainerDied","Data":"dffc05feeecacd8fb59242cef52027e30813e6e0daeafd2db6f17bc5381db69c"} Jul 13 20:10:12.325623 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.325597 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475398704" Jul 13 20:10:12.345817 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.345727 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7702-139685475398704_6677d4b5-764f-4be0-bf5c-c70e1fd185d5/test-curl-7702-139685475398704/0.log" Jul 13 20:10:12.367981 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.367936 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7702-139685475398704"] Jul 13 20:10:12.371486 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.371449 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7702-139685475398704"] Jul 13 20:10:12.458637 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.458529 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4l759\" (UniqueName: \"kubernetes.io/projected/6677d4b5-764f-4be0-bf5c-c70e1fd185d5-kube-api-access-4l759\") pod \"6677d4b5-764f-4be0-bf5c-c70e1fd185d5\" (UID: \"6677d4b5-764f-4be0-bf5c-c70e1fd185d5\") " Jul 13 20:10:12.460955 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.460919 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6677d4b5-764f-4be0-bf5c-c70e1fd185d5-kube-api-access-4l759" (OuterVolumeSpecName: "kube-api-access-4l759") pod "6677d4b5-764f-4be0-bf5c-c70e1fd185d5" (UID: "6677d4b5-764f-4be0-bf5c-c70e1fd185d5"). InnerVolumeSpecName "kube-api-access-4l759". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:10:12.522583 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.522544 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7702-139685475590192"] Jul 13 20:10:12.523028 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.523014 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6677d4b5-764f-4be0-bf5c-c70e1fd185d5" containerName="test-curl-7702-139685475398704" Jul 13 20:10:12.523071 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.523030 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6677d4b5-764f-4be0-bf5c-c70e1fd185d5" containerName="test-curl-7702-139685475398704" Jul 13 20:10:12.523135 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.523125 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6677d4b5-764f-4be0-bf5c-c70e1fd185d5" containerName="test-curl-7702-139685475398704" Jul 13 20:10:12.526483 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.526464 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475590192" Jul 13 20:10:12.532614 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.532583 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7702-139685475590192"] Jul 13 20:10:12.559881 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.559837 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4l759\" (UniqueName: \"kubernetes.io/projected/6677d4b5-764f-4be0-bf5c-c70e1fd185d5-kube-api-access-4l759\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:10:12.660349 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.660300 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4dxq\" (UniqueName: \"kubernetes.io/projected/b6f8c5fb-46c0-4208-bdc0-877ddb758bdd-kube-api-access-h4dxq\") pod \"test-curl-7702-139685475590192\" (UID: \"b6f8c5fb-46c0-4208-bdc0-877ddb758bdd\") " pod="openshift-ingress/test-curl-7702-139685475590192" Jul 13 20:10:12.761355 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.761249 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4dxq\" (UniqueName: \"kubernetes.io/projected/b6f8c5fb-46c0-4208-bdc0-877ddb758bdd-kube-api-access-h4dxq\") pod \"test-curl-7702-139685475590192\" (UID: \"b6f8c5fb-46c0-4208-bdc0-877ddb758bdd\") " pod="openshift-ingress/test-curl-7702-139685475590192" Jul 13 20:10:12.769009 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.768971 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4dxq\" (UniqueName: \"kubernetes.io/projected/b6f8c5fb-46c0-4208-bdc0-877ddb758bdd-kube-api-access-h4dxq\") pod \"test-curl-7702-139685475590192\" (UID: \"b6f8c5fb-46c0-4208-bdc0-877ddb758bdd\") " pod="openshift-ingress/test-curl-7702-139685475590192" Jul 13 20:10:12.837724 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.837687 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475590192" Jul 13 20:10:12.851644 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.851609 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6677d4b5-764f-4be0-bf5c-c70e1fd185d5" path="/var/lib/kubelet/pods/6677d4b5-764f-4be0-bf5c-c70e1fd185d5/volumes" Jul 13 20:10:12.986903 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:12.986868 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7702-139685475590192"] Jul 13 20:10:12.989512 ip-10-0-132-20 kubenswrapper[2556]: W0713 20:10:12.989476 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6f8c5fb_46c0_4208_bdc0_877ddb758bdd.slice/crio-c8a804f69b2801e5c9a727aaeff705ec6ddcd73a5f870bc474565798ea6b49a4 WatchSource:0}: Error finding container c8a804f69b2801e5c9a727aaeff705ec6ddcd73a5f870bc474565798ea6b49a4: Status 404 returned error can't find the container with id c8a804f69b2801e5c9a727aaeff705ec6ddcd73a5f870bc474565798ea6b49a4 Jul 13 20:10:13.180747 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:13.180712 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475398704" Jul 13 20:10:13.180747 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:13.180722 2556 scope.go:117] "RemoveContainer" containerID="dffc05feeecacd8fb59242cef52027e30813e6e0daeafd2db6f17bc5381db69c" Jul 13 20:10:13.182205 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:13.182011 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7702-139685475590192" event={"ID":"b6f8c5fb-46c0-4208-bdc0-877ddb758bdd","Type":"ContainerStarted","Data":"c8a804f69b2801e5c9a727aaeff705ec6ddcd73a5f870bc474565798ea6b49a4"} Jul 13 20:10:14.190514 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:14.190473 2556 generic.go:356] "Generic (PLEG): container finished" podID="b6f8c5fb-46c0-4208-bdc0-877ddb758bdd" containerID="3ea65b9ab7f1e1fa5a94f6045b5310eb1cd66233ada48ba4381f4faf25850a7b" exitCode=0 Jul 13 20:10:14.191007 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:14.190562 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7702-139685475590192" event={"ID":"b6f8c5fb-46c0-4208-bdc0-877ddb758bdd","Type":"ContainerDied","Data":"3ea65b9ab7f1e1fa5a94f6045b5310eb1cd66233ada48ba4381f4faf25850a7b"} Jul 13 20:10:15.352249 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:15.352221 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475590192" Jul 13 20:10:15.374733 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:15.374687 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7702-139685475590192_b6f8c5fb-46c0-4208-bdc0-877ddb758bdd/test-curl-7702-139685475590192/0.log" Jul 13 20:10:15.399612 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:15.399579 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7702-139685475590192"] Jul 13 20:10:15.407002 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:15.406974 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7702-139685475590192"] Jul 13 20:10:15.490734 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:15.490633 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4dxq\" (UniqueName: \"kubernetes.io/projected/b6f8c5fb-46c0-4208-bdc0-877ddb758bdd-kube-api-access-h4dxq\") pod \"b6f8c5fb-46c0-4208-bdc0-877ddb758bdd\" (UID: \"b6f8c5fb-46c0-4208-bdc0-877ddb758bdd\") " Jul 13 20:10:15.492917 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:15.492882 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6f8c5fb-46c0-4208-bdc0-877ddb758bdd-kube-api-access-h4dxq" (OuterVolumeSpecName: "kube-api-access-h4dxq") pod "b6f8c5fb-46c0-4208-bdc0-877ddb758bdd" (UID: "b6f8c5fb-46c0-4208-bdc0-877ddb758bdd"). InnerVolumeSpecName "kube-api-access-h4dxq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:10:15.526549 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:15.526514 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7702-139685475590192"] Jul 13 20:10:15.527000 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:15.526985 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f8c5fb-46c0-4208-bdc0-877ddb758bdd" containerName="test-curl-7702-139685475590192" Jul 13 20:10:15.527043 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:15.527001 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f8c5fb-46c0-4208-bdc0-877ddb758bdd" containerName="test-curl-7702-139685475590192" Jul 13 20:10:15.527098 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:15.527084 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6f8c5fb-46c0-4208-bdc0-877ddb758bdd" containerName="test-curl-7702-139685475590192" Jul 13 20:10:15.530194 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:15.530175 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475590192" Jul 13 20:10:15.535366 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:15.535325 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7702-139685475590192"] Jul 13 20:10:15.591536 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:15.591502 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxt2p\" (UniqueName: \"kubernetes.io/projected/402a153b-ad78-4f80-b805-89ecc0450ac3-kube-api-access-wxt2p\") pod \"test-curl-7702-139685475590192\" (UID: \"402a153b-ad78-4f80-b805-89ecc0450ac3\") " pod="openshift-ingress/test-curl-7702-139685475590192" Jul 13 20:10:15.591713 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:15.591569 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4dxq\" (UniqueName: \"kubernetes.io/projected/b6f8c5fb-46c0-4208-bdc0-877ddb758bdd-kube-api-access-h4dxq\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:10:15.692910 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:15.692870 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxt2p\" (UniqueName: \"kubernetes.io/projected/402a153b-ad78-4f80-b805-89ecc0450ac3-kube-api-access-wxt2p\") pod \"test-curl-7702-139685475590192\" (UID: \"402a153b-ad78-4f80-b805-89ecc0450ac3\") " pod="openshift-ingress/test-curl-7702-139685475590192" Jul 13 20:10:15.701543 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:15.701502 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxt2p\" (UniqueName: \"kubernetes.io/projected/402a153b-ad78-4f80-b805-89ecc0450ac3-kube-api-access-wxt2p\") pod \"test-curl-7702-139685475590192\" (UID: \"402a153b-ad78-4f80-b805-89ecc0450ac3\") " pod="openshift-ingress/test-curl-7702-139685475590192" Jul 13 20:10:15.841731 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:15.841690 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475590192" Jul 13 20:10:15.989630 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:15.989601 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7702-139685475590192"] Jul 13 20:10:15.991128 ip-10-0-132-20 kubenswrapper[2556]: W0713 20:10:15.991094 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod402a153b_ad78_4f80_b805_89ecc0450ac3.slice/crio-baa098f236175e966315d69aad09f917ef3508a41ba33d00c3d840adfa56f2d2 WatchSource:0}: Error finding container baa098f236175e966315d69aad09f917ef3508a41ba33d00c3d840adfa56f2d2: Status 404 returned error can't find the container with id baa098f236175e966315d69aad09f917ef3508a41ba33d00c3d840adfa56f2d2 Jul 13 20:10:16.204064 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:16.203953 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7702-139685475590192" event={"ID":"402a153b-ad78-4f80-b805-89ecc0450ac3","Type":"ContainerStarted","Data":"baa098f236175e966315d69aad09f917ef3508a41ba33d00c3d840adfa56f2d2"} Jul 13 20:10:16.205495 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:16.205474 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475590192" Jul 13 20:10:16.205625 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:16.205479 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8a804f69b2801e5c9a727aaeff705ec6ddcd73a5f870bc474565798ea6b49a4" Jul 13 20:10:16.218195 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:16.218154 2556 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7702-139685475590192" oldPodUID="b6f8c5fb-46c0-4208-bdc0-877ddb758bdd" podUID="402a153b-ad78-4f80-b805-89ecc0450ac3" Jul 13 20:10:16.850060 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:16.850025 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6f8c5fb-46c0-4208-bdc0-877ddb758bdd" path="/var/lib/kubelet/pods/b6f8c5fb-46c0-4208-bdc0-877ddb758bdd/volumes" Jul 13 20:10:16.903155 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:16.903115 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:10:17.213142 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:17.213057 2556 generic.go:356] "Generic (PLEG): container finished" podID="402a153b-ad78-4f80-b805-89ecc0450ac3" containerID="0961e2e02755b29c5397a0c2902d74d089f1c320a2565795a247c955109bd93d" exitCode=0 Jul 13 20:10:17.213285 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:17.213142 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7702-139685475590192" event={"ID":"402a153b-ad78-4f80-b805-89ecc0450ac3","Type":"ContainerDied","Data":"0961e2e02755b29c5397a0c2902d74d089f1c320a2565795a247c955109bd93d"} Jul 13 20:10:18.387954 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:18.387922 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475590192" Jul 13 20:10:18.408930 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:18.408900 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7702-139685475590192_402a153b-ad78-4f80-b805-89ecc0450ac3/test-curl-7702-139685475590192/0.log" Jul 13 20:10:18.435003 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:18.434952 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7702-139685475590192"] Jul 13 20:10:18.440235 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:18.440196 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7702-139685475590192"] Jul 13 20:10:18.524251 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:18.524151 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxt2p\" (UniqueName: \"kubernetes.io/projected/402a153b-ad78-4f80-b805-89ecc0450ac3-kube-api-access-wxt2p\") pod \"402a153b-ad78-4f80-b805-89ecc0450ac3\" (UID: \"402a153b-ad78-4f80-b805-89ecc0450ac3\") " Jul 13 20:10:18.526411 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:18.526374 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/402a153b-ad78-4f80-b805-89ecc0450ac3-kube-api-access-wxt2p" (OuterVolumeSpecName: "kube-api-access-wxt2p") pod "402a153b-ad78-4f80-b805-89ecc0450ac3" (UID: "402a153b-ad78-4f80-b805-89ecc0450ac3"). InnerVolumeSpecName "kube-api-access-wxt2p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:10:18.561296 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:18.561252 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7702-139685475591200"] Jul 13 20:10:18.561803 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:18.561787 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="402a153b-ad78-4f80-b805-89ecc0450ac3" containerName="test-curl-7702-139685475590192" Jul 13 20:10:18.561846 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:18.561804 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="402a153b-ad78-4f80-b805-89ecc0450ac3" containerName="test-curl-7702-139685475590192" Jul 13 20:10:18.561884 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:18.561873 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="402a153b-ad78-4f80-b805-89ecc0450ac3" containerName="test-curl-7702-139685475590192" Jul 13 20:10:18.565058 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:18.565035 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475591200" Jul 13 20:10:18.570946 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:18.570916 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7702-139685475591200"] Jul 13 20:10:18.625689 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:18.625655 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxt2p\" (UniqueName: \"kubernetes.io/projected/402a153b-ad78-4f80-b805-89ecc0450ac3-kube-api-access-wxt2p\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:10:18.727211 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:18.727174 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zb77\" (UniqueName: \"kubernetes.io/projected/23576f99-61cb-41f2-9a35-89f53403830b-kube-api-access-5zb77\") pod \"test-curl-7702-139685475591200\" (UID: \"23576f99-61cb-41f2-9a35-89f53403830b\") " pod="openshift-ingress/test-curl-7702-139685475591200" Jul 13 20:10:18.828394 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:18.828350 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zb77\" (UniqueName: \"kubernetes.io/projected/23576f99-61cb-41f2-9a35-89f53403830b-kube-api-access-5zb77\") pod \"test-curl-7702-139685475591200\" (UID: \"23576f99-61cb-41f2-9a35-89f53403830b\") " pod="openshift-ingress/test-curl-7702-139685475591200" Jul 13 20:10:18.837177 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:18.837145 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zb77\" (UniqueName: \"kubernetes.io/projected/23576f99-61cb-41f2-9a35-89f53403830b-kube-api-access-5zb77\") pod \"test-curl-7702-139685475591200\" (UID: \"23576f99-61cb-41f2-9a35-89f53403830b\") " pod="openshift-ingress/test-curl-7702-139685475591200" Jul 13 20:10:18.850293 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:18.850252 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="402a153b-ad78-4f80-b805-89ecc0450ac3" path="/var/lib/kubelet/pods/402a153b-ad78-4f80-b805-89ecc0450ac3/volumes" Jul 13 20:10:18.876744 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:18.876704 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475591200" Jul 13 20:10:19.029205 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:19.029171 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7702-139685475591200"] Jul 13 20:10:19.030935 ip-10-0-132-20 kubenswrapper[2556]: W0713 20:10:19.030904 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23576f99_61cb_41f2_9a35_89f53403830b.slice/crio-b2edcc629121920951ee584c8d128e02d984cbe81b9729a188e0596e65c26b27 WatchSource:0}: Error finding container b2edcc629121920951ee584c8d128e02d984cbe81b9729a188e0596e65c26b27: Status 404 returned error can't find the container with id b2edcc629121920951ee584c8d128e02d984cbe81b9729a188e0596e65c26b27 Jul 13 20:10:19.227504 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:19.227410 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475590192" Jul 13 20:10:19.227504 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:19.227467 2556 scope.go:117] "RemoveContainer" containerID="0961e2e02755b29c5397a0c2902d74d089f1c320a2565795a247c955109bd93d" Jul 13 20:10:19.228992 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:19.228962 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7702-139685475591200" event={"ID":"23576f99-61cb-41f2-9a35-89f53403830b","Type":"ContainerStarted","Data":"b2edcc629121920951ee584c8d128e02d984cbe81b9729a188e0596e65c26b27"} Jul 13 20:10:20.238638 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:20.238601 2556 generic.go:356] "Generic (PLEG): container finished" podID="23576f99-61cb-41f2-9a35-89f53403830b" containerID="2c8a0722c38684d7fed5ab8c09f1a9ec8b57d27a4258e5802b4b35d2ee601e57" exitCode=0 Jul 13 20:10:20.239054 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:20.238688 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7702-139685475591200" event={"ID":"23576f99-61cb-41f2-9a35-89f53403830b","Type":"ContainerDied","Data":"2c8a0722c38684d7fed5ab8c09f1a9ec8b57d27a4258e5802b4b35d2ee601e57"} Jul 13 20:10:21.395079 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:21.395049 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475591200" Jul 13 20:10:21.415532 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:21.415503 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7702-139685475591200_23576f99-61cb-41f2-9a35-89f53403830b/test-curl-7702-139685475591200/0.log" Jul 13 20:10:21.440443 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:21.440397 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7702-139685475591200"] Jul 13 20:10:21.445575 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:21.445541 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7702-139685475591200"] Jul 13 20:10:21.555438 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:21.555404 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zb77\" (UniqueName: \"kubernetes.io/projected/23576f99-61cb-41f2-9a35-89f53403830b-kube-api-access-5zb77\") pod \"23576f99-61cb-41f2-9a35-89f53403830b\" (UID: \"23576f99-61cb-41f2-9a35-89f53403830b\") " Jul 13 20:10:21.557743 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:21.557704 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23576f99-61cb-41f2-9a35-89f53403830b-kube-api-access-5zb77" (OuterVolumeSpecName: "kube-api-access-5zb77") pod "23576f99-61cb-41f2-9a35-89f53403830b" (UID: "23576f99-61cb-41f2-9a35-89f53403830b"). InnerVolumeSpecName "kube-api-access-5zb77". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:10:21.656810 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:21.656734 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zb77\" (UniqueName: \"kubernetes.io/projected/23576f99-61cb-41f2-9a35-89f53403830b-kube-api-access-5zb77\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:10:22.254244 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:22.254206 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7702-139685475591200" Jul 13 20:10:22.254244 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:22.254242 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2edcc629121920951ee584c8d128e02d984cbe81b9729a188e0596e65c26b27" Jul 13 20:10:22.267935 ip-10-0-132-20 kubenswrapper[2556]: E0713 20:10:22.267883 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7702-139685475591200\" is forbidden: User \"system:node:ip-10-0-132-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-132-20.ec2.internal' and this object" podUID="23576f99-61cb-41f2-9a35-89f53403830b" pod="openshift-ingress/test-curl-7702-139685475591200" Jul 13 20:10:22.849429 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:22.849385 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23576f99-61cb-41f2-9a35-89f53403830b" path="/var/lib/kubelet/pods/23576f99-61cb-41f2-9a35-89f53403830b/volumes" Jul 13 20:10:35.216293 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:35.216252 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds"] Jul 13 20:10:35.216968 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:35.216566 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" podUID="1ebed944-88fa-4bbc-ad43-8dbf532e972e" containerName="maas-api" containerID="cri-o://21ef26b58efe1e4544a60c2f7ca3cc81e4d0003fca61b1f9d2447b65dab45400" gracePeriod=30 Jul 13 20:10:35.338815 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:35.338758 2556 generic.go:356] "Generic (PLEG): container finished" podID="1ebed944-88fa-4bbc-ad43-8dbf532e972e" containerID="21ef26b58efe1e4544a60c2f7ca3cc81e4d0003fca61b1f9d2447b65dab45400" exitCode=0 Jul 13 20:10:35.338978 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:35.338809 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" event={"ID":"1ebed944-88fa-4bbc-ad43-8dbf532e972e","Type":"ContainerDied","Data":"21ef26b58efe1e4544a60c2f7ca3cc81e4d0003fca61b1f9d2447b65dab45400"} Jul 13 20:10:35.508262 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:35.508232 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" Jul 13 20:10:35.603093 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:35.603049 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ebed944-88fa-4bbc-ad43-8dbf532e972e-openshift-service-ca\") pod \"1ebed944-88fa-4bbc-ad43-8dbf532e972e\" (UID: \"1ebed944-88fa-4bbc-ad43-8dbf532e972e\") " Jul 13 20:10:35.603292 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:35.603156 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29cx2\" (UniqueName: \"kubernetes.io/projected/1ebed944-88fa-4bbc-ad43-8dbf532e972e-kube-api-access-29cx2\") pod \"1ebed944-88fa-4bbc-ad43-8dbf532e972e\" (UID: \"1ebed944-88fa-4bbc-ad43-8dbf532e972e\") " Jul 13 20:10:35.603292 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:35.603258 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ebed944-88fa-4bbc-ad43-8dbf532e972e-maas-api-tls\") pod \"1ebed944-88fa-4bbc-ad43-8dbf532e972e\" (UID: \"1ebed944-88fa-4bbc-ad43-8dbf532e972e\") " Jul 13 20:10:35.603474 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:35.603448 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ebed944-88fa-4bbc-ad43-8dbf532e972e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1ebed944-88fa-4bbc-ad43-8dbf532e972e" (UID: "1ebed944-88fa-4bbc-ad43-8dbf532e972e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:10:35.603629 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:35.603612 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ebed944-88fa-4bbc-ad43-8dbf532e972e-openshift-service-ca\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:10:35.605364 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:35.605336 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ebed944-88fa-4bbc-ad43-8dbf532e972e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1ebed944-88fa-4bbc-ad43-8dbf532e972e" (UID: "1ebed944-88fa-4bbc-ad43-8dbf532e972e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:10:35.605504 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:35.605485 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ebed944-88fa-4bbc-ad43-8dbf532e972e-kube-api-access-29cx2" (OuterVolumeSpecName: "kube-api-access-29cx2") pod "1ebed944-88fa-4bbc-ad43-8dbf532e972e" (UID: "1ebed944-88fa-4bbc-ad43-8dbf532e972e"). InnerVolumeSpecName "kube-api-access-29cx2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:10:35.704703 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:35.704656 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ebed944-88fa-4bbc-ad43-8dbf532e972e-maas-api-tls\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:10:35.704703 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:35.704695 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29cx2\" (UniqueName: \"kubernetes.io/projected/1ebed944-88fa-4bbc-ad43-8dbf532e972e-kube-api-access-29cx2\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:10:36.348113 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:36.348077 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" Jul 13 20:10:36.348640 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:36.348108 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds" event={"ID":"1ebed944-88fa-4bbc-ad43-8dbf532e972e","Type":"ContainerDied","Data":"fb0678509b467214cd39be738f4b1a938b655bd66f341fd06e83f841c7b9826c"} Jul 13 20:10:36.348640 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:36.348185 2556 scope.go:117] "RemoveContainer" containerID="21ef26b58efe1e4544a60c2f7ca3cc81e4d0003fca61b1f9d2447b65dab45400" Jul 13 20:10:36.373282 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:36.373241 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds"] Jul 13 20:10:36.377160 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:36.377128 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-452240-5dc69587b8-ttbds"] Jul 13 20:10:36.850426 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:36.850393 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ebed944-88fa-4bbc-ad43-8dbf532e972e" path="/var/lib/kubelet/pods/1ebed944-88fa-4bbc-ad43-8dbf532e972e/volumes" Jul 13 20:10:39.032637 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:10:39.032593 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:11:01.350030 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:01.349986 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q"] Jul 13 20:11:01.350572 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:01.350247 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" podUID="5d3e1560-2993-4412-a65f-7092a067ed2e" containerName="maas-api" containerID="cri-o://e20648baaf7c197736ef58a4810fca6849575e2ab87c7e5382084ea4b81b32d7" gracePeriod=30 Jul 13 20:11:01.504711 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:01.504662 2556 generic.go:356] "Generic (PLEG): container finished" podID="5d3e1560-2993-4412-a65f-7092a067ed2e" containerID="e20648baaf7c197736ef58a4810fca6849575e2ab87c7e5382084ea4b81b32d7" exitCode=0 Jul 13 20:11:01.504935 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:01.504727 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" event={"ID":"5d3e1560-2993-4412-a65f-7092a067ed2e","Type":"ContainerDied","Data":"e20648baaf7c197736ef58a4810fca6849575e2ab87c7e5382084ea4b81b32d7"} Jul 13 20:11:01.618709 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:01.618676 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" Jul 13 20:11:01.768409 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:01.768358 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d3e1560-2993-4412-a65f-7092a067ed2e-openshift-service-ca\") pod \"5d3e1560-2993-4412-a65f-7092a067ed2e\" (UID: \"5d3e1560-2993-4412-a65f-7092a067ed2e\") " Jul 13 20:11:01.768619 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:01.768425 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmp5b\" (UniqueName: \"kubernetes.io/projected/5d3e1560-2993-4412-a65f-7092a067ed2e-kube-api-access-kmp5b\") pod \"5d3e1560-2993-4412-a65f-7092a067ed2e\" (UID: \"5d3e1560-2993-4412-a65f-7092a067ed2e\") " Jul 13 20:11:01.768619 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:01.768459 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d3e1560-2993-4412-a65f-7092a067ed2e-maas-api-tls\") pod \"5d3e1560-2993-4412-a65f-7092a067ed2e\" (UID: \"5d3e1560-2993-4412-a65f-7092a067ed2e\") " Jul 13 20:11:01.768857 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:01.768832 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d3e1560-2993-4412-a65f-7092a067ed2e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5d3e1560-2993-4412-a65f-7092a067ed2e" (UID: "5d3e1560-2993-4412-a65f-7092a067ed2e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:11:01.770724 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:01.770694 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d3e1560-2993-4412-a65f-7092a067ed2e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5d3e1560-2993-4412-a65f-7092a067ed2e" (UID: "5d3e1560-2993-4412-a65f-7092a067ed2e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:11:01.770870 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:01.770760 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d3e1560-2993-4412-a65f-7092a067ed2e-kube-api-access-kmp5b" (OuterVolumeSpecName: "kube-api-access-kmp5b") pod "5d3e1560-2993-4412-a65f-7092a067ed2e" (UID: "5d3e1560-2993-4412-a65f-7092a067ed2e"). InnerVolumeSpecName "kube-api-access-kmp5b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:11:01.869583 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:01.869480 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d3e1560-2993-4412-a65f-7092a067ed2e-openshift-service-ca\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:11:01.869583 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:01.869524 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmp5b\" (UniqueName: \"kubernetes.io/projected/5d3e1560-2993-4412-a65f-7092a067ed2e-kube-api-access-kmp5b\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:11:01.869583 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:01.869540 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d3e1560-2993-4412-a65f-7092a067ed2e-maas-api-tls\") on node \"ip-10-0-132-20.ec2.internal\" DevicePath \"\"" Jul 13 20:11:02.511537 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:02.511506 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" Jul 13 20:11:02.511993 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:02.511499 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q" event={"ID":"5d3e1560-2993-4412-a65f-7092a067ed2e","Type":"ContainerDied","Data":"510a36cbcb4dc47eb377cc8f46079bfc24a5a0a5d4be50bf1d0c7a5be1d5d57a"} Jul 13 20:11:02.511993 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:02.511646 2556 scope.go:117] "RemoveContainer" containerID="e20648baaf7c197736ef58a4810fca6849575e2ab87c7e5382084ea4b81b32d7" Jul 13 20:11:02.539202 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:02.539171 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q"] Jul 13 20:11:02.549016 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:02.548986 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-83eaac-67f8ffdb87-g248q"] Jul 13 20:11:02.850608 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:02.850571 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d3e1560-2993-4412-a65f-7092a067ed2e" path="/var/lib/kubelet/pods/5d3e1560-2993-4412-a65f-7092a067ed2e/volumes" Jul 13 20:11:04.614395 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:04.614350 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tcvzv"] Jul 13 20:11:22.226122 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:22.226039 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-f5pvv_532f7bb2-91cc-4a13-9e97-7c379750744f/manager/0.log" Jul 13 20:11:22.997878 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:22.997846 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6996d85cfb-j9wjq_de2480a1-3f58-4a81-835f-1ee281d2ba2e/copy-manifests/0.log" Jul 13 20:11:23.004091 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:23.004068 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6996d85cfb-j9wjq_de2480a1-3f58-4a81-835f-1ee281d2ba2e/manager/0.log" Jul 13 20:11:24.701406 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:24.701378 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-d67d76ca-5fb9j_40814be3-8200-431a-a592-c1bfe7cb53c6/revoke-keys/0.log" Jul 13 20:11:25.841080 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:25.841044 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn_b754ecef-c6d5-40b8-8bc8-5a0f4cee389e/pull/0.log" Jul 13 20:11:25.847364 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:25.847328 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn_b754ecef-c6d5-40b8-8bc8-5a0f4cee389e/extract/0.log" Jul 13 20:11:25.853175 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:25.853153 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn_b754ecef-c6d5-40b8-8bc8-5a0f4cee389e/util/0.log" Jul 13 20:11:25.975294 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:25.975263 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms_1a6ed951-de47-4ce7-a425-5704fdea97b6/util/0.log" Jul 13 20:11:25.981075 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:25.981051 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms_1a6ed951-de47-4ce7-a425-5704fdea97b6/pull/0.log" Jul 13 20:11:25.988477 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:25.988442 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms_1a6ed951-de47-4ce7-a425-5704fdea97b6/extract/0.log" Jul 13 20:11:26.106423 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:26.106339 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s_0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4/util/0.log" Jul 13 20:11:26.114115 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:26.114081 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s_0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4/pull/0.log" Jul 13 20:11:26.120962 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:26.120933 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s_0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4/extract/0.log" Jul 13 20:11:26.236512 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:26.236484 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh_5cbf56db-28a7-43d4-a29b-a1ec5ff0371d/util/0.log" Jul 13 20:11:26.243239 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:26.243214 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh_5cbf56db-28a7-43d4-a29b-a1ec5ff0371d/pull/0.log" Jul 13 20:11:26.249342 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:26.249319 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh_5cbf56db-28a7-43d4-a29b-a1ec5ff0371d/extract/0.log" Jul 13 20:11:26.608002 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:26.607972 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-x9bzx_c84f73db-a1da-4027-9a6e-bdc3cd0ad6b8/manager/0.log" Jul 13 20:11:26.972107 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:26.972018 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-wn6b6_05b2672c-c7b1-4b12-8418-6db7049250ce/manager/0.log" Jul 13 20:11:27.095388 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:27.095360 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-tcvzv_b0f25a86-9f3a-4451-b67f-8adada658cd6/limitador/0.log" Jul 13 20:11:27.557011 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:27.556979 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mc4gc_a890d805-f6dc-483b-aaaa-8e6bfe48ef08/istio-proxy/0.log" Jul 13 20:11:28.016679 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:28.016586 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-l8mnm_16e74623-f867-449e-a739-4234322bb3e1/istio-proxy/1.log" Jul 13 20:11:28.979346 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:28.979312 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-fnwrq_b8ad2db9-f086-4c2f-8489-b5ef9a79b944/main/0.log" Jul 13 20:11:36.222188 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:36.222156 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qgb5b_5acbdd58-54bb-4a1e-a631-0ced4ea6c5d4/global-pull-secret-syncer/0.log" Jul 13 20:11:36.374166 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:36.374132 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wzrvz_bd6bcf64-ab9b-4c5f-b23b-07f454be4c24/konnectivity-agent/0.log" Jul 13 20:11:36.449326 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:36.449291 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-20.ec2.internal_e49ce7abc689f72d6258a00fc408109c/haproxy/0.log" Jul 13 20:11:41.827766 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:41.827737 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn_b754ecef-c6d5-40b8-8bc8-5a0f4cee389e/extract/0.log" Jul 13 20:11:41.862337 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:41.862304 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn_b754ecef-c6d5-40b8-8bc8-5a0f4cee389e/util/0.log" Jul 13 20:11:41.903581 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:41.903550 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l8bzn_b754ecef-c6d5-40b8-8bc8-5a0f4cee389e/pull/0.log" Jul 13 20:11:41.946026 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:41.945994 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms_1a6ed951-de47-4ce7-a425-5704fdea97b6/extract/0.log" Jul 13 20:11:41.972652 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:41.972621 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms_1a6ed951-de47-4ce7-a425-5704fdea97b6/util/0.log" Jul 13 20:11:41.997112 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:41.997072 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0h4bms_1a6ed951-de47-4ce7-a425-5704fdea97b6/pull/0.log" Jul 13 20:11:42.028130 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:42.028086 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s_0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4/extract/0.log" Jul 13 20:11:42.052080 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:42.052050 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s_0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4/util/0.log" Jul 13 20:11:42.076102 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:42.076070 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nmv9s_0a69d81d-09ea-4c30-9aff-bc5bb77f0ee4/pull/0.log" Jul 13 20:11:42.108953 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:42.108845 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh_5cbf56db-28a7-43d4-a29b-a1ec5ff0371d/extract/0.log" Jul 13 20:11:42.141719 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:42.141689 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh_5cbf56db-28a7-43d4-a29b-a1ec5ff0371d/util/0.log" Jul 13 20:11:42.168584 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:42.168536 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wxjmh_5cbf56db-28a7-43d4-a29b-a1ec5ff0371d/pull/0.log" Jul 13 20:11:42.851926 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:42.851888 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-x9bzx_c84f73db-a1da-4027-9a6e-bdc3cd0ad6b8/manager/0.log" Jul 13 20:11:43.127202 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:43.127104 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-wn6b6_05b2672c-c7b1-4b12-8418-6db7049250ce/manager/0.log" Jul 13 20:11:43.183577 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:43.183542 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-tcvzv_b0f25a86-9f3a-4451-b67f-8adada658cd6/limitador/0.log" Jul 13 20:11:45.023676 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:45.023562 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43da95a6-a681-46f7-902e-6e206d0f299f/alertmanager/0.log" Jul 13 20:11:45.051123 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:45.051091 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43da95a6-a681-46f7-902e-6e206d0f299f/config-reloader/0.log" Jul 13 20:11:45.079896 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:45.079860 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43da95a6-a681-46f7-902e-6e206d0f299f/kube-rbac-proxy-web/0.log" Jul 13 20:11:45.108228 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:45.108202 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43da95a6-a681-46f7-902e-6e206d0f299f/kube-rbac-proxy/0.log" Jul 13 20:11:45.139421 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:45.139391 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43da95a6-a681-46f7-902e-6e206d0f299f/kube-rbac-proxy-metric/0.log" Jul 13 20:11:45.164587 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:45.164553 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43da95a6-a681-46f7-902e-6e206d0f299f/prom-label-proxy/0.log" Jul 13 20:11:45.192554 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:45.192528 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43da95a6-a681-46f7-902e-6e206d0f299f/init-config-reloader/0.log" Jul 13 20:11:45.286022 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:45.285992 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-c8pz6_c04a5ce2-c2ae-4036-9a7c-519c30553a48/kube-state-metrics/0.log" Jul 13 20:11:45.312600 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:45.312566 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-c8pz6_c04a5ce2-c2ae-4036-9a7c-519c30553a48/kube-rbac-proxy-main/0.log" Jul 13 20:11:45.346760 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:45.346729 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-c8pz6_c04a5ce2-c2ae-4036-9a7c-519c30553a48/kube-rbac-proxy-self/0.log" Jul 13 20:11:45.382023 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:45.381992 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7669c847b-whh8v_334df410-3d09-4551-87e8-4f7184fdba55/metrics-server/0.log" Jul 13 20:11:45.410679 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:45.410646 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-pnb99_a7bdc7c5-be2c-4135-a6d2-1f901510cf6d/monitoring-plugin/0.log" Jul 13 20:11:45.554378 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:45.554290 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qg567_e4aa6015-fd26-40b5-abfe-d3df70e1c058/node-exporter/0.log" Jul 13 20:11:45.590748 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:45.590717 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qg567_e4aa6015-fd26-40b5-abfe-d3df70e1c058/kube-rbac-proxy/0.log" Jul 13 20:11:45.641094 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:45.641063 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qg567_e4aa6015-fd26-40b5-abfe-d3df70e1c058/init-textfile/0.log" Jul 13 20:11:45.770512 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:45.770478 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-2gxgn_39da47d2-4e42-4d22-955e-10adfc404b57/kube-rbac-proxy-main/0.log" Jul 13 20:11:45.820070 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:45.819986 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-2gxgn_39da47d2-4e42-4d22-955e-10adfc404b57/kube-rbac-proxy-self/0.log" Jul 13 20:11:45.846674 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:45.846641 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-2gxgn_39da47d2-4e42-4d22-955e-10adfc404b57/openshift-state-metrics/0.log" Jul 13 20:11:46.098609 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:46.098505 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-knl5f_e2d8ed68-6996-4c40-9f6a-7a03384fb5b1/prometheus-operator/0.log" Jul 13 20:11:46.124831 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:46.124800 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-knl5f_e2d8ed68-6996-4c40-9f6a-7a03384fb5b1/kube-rbac-proxy/0.log" Jul 13 20:11:46.157551 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:46.157520 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-xlhtc_7b87aa78-a616-4642-a245-108f3baabde8/prometheus-operator-admission-webhook/0.log" Jul 13 20:11:46.191680 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:46.191650 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 20:11:46.195523 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:46.195466 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-656664b677-6trm6_75d3158d-8e01-4a6c-8ca9-66d0a2f1077a/telemeter-client/0.log" Jul 13 20:11:46.200940 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:46.200914 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bk46s_a72fd793-62c4-49b9-b3cb-06d08aa88ec7/ovn-acl-logging/0.log" Jul 13 20:11:46.240895 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:46.240866 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-656664b677-6trm6_75d3158d-8e01-4a6c-8ca9-66d0a2f1077a/reload/0.log" Jul 13 20:11:46.282275 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:46.282236 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-656664b677-6trm6_75d3158d-8e01-4a6c-8ca9-66d0a2f1077a/kube-rbac-proxy/0.log" Jul 13 20:11:46.316130 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:46.316100 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d9499477c-2lz6w_08923444-e4b3-4cdb-8bba-fe70a9b5c218/thanos-query/0.log" Jul 13 20:11:46.339890 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:46.339862 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d9499477c-2lz6w_08923444-e4b3-4cdb-8bba-fe70a9b5c218/kube-rbac-proxy-web/0.log" Jul 13 20:11:46.361904 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:46.361826 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d9499477c-2lz6w_08923444-e4b3-4cdb-8bba-fe70a9b5c218/kube-rbac-proxy/0.log" Jul 13 20:11:46.384326 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:46.384295 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d9499477c-2lz6w_08923444-e4b3-4cdb-8bba-fe70a9b5c218/prom-label-proxy/0.log" Jul 13 20:11:46.407505 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:46.407473 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d9499477c-2lz6w_08923444-e4b3-4cdb-8bba-fe70a9b5c218/kube-rbac-proxy-rules/0.log" Jul 13 20:11:46.434312 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:46.434281 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d9499477c-2lz6w_08923444-e4b3-4cdb-8bba-fe70a9b5c218/kube-rbac-proxy-metrics/0.log" Jul 13 20:11:47.670960 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.670920 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58"] Jul 13 20:11:47.671486 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.671403 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ebed944-88fa-4bbc-ad43-8dbf532e972e" containerName="maas-api" Jul 13 20:11:47.671486 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.671417 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ebed944-88fa-4bbc-ad43-8dbf532e972e" containerName="maas-api" Jul 13 20:11:47.671486 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.671433 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23576f99-61cb-41f2-9a35-89f53403830b" containerName="test-curl-7702-139685475591200" Jul 13 20:11:47.671486 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.671441 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="23576f99-61cb-41f2-9a35-89f53403830b" containerName="test-curl-7702-139685475591200" Jul 13 20:11:47.671486 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.671471 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d3e1560-2993-4412-a65f-7092a067ed2e" containerName="maas-api" Jul 13 20:11:47.671486 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.671478 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3e1560-2993-4412-a65f-7092a067ed2e" containerName="maas-api" Jul 13 20:11:47.671686 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.671540 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d3e1560-2993-4412-a65f-7092a067ed2e" containerName="maas-api" Jul 13 20:11:47.671686 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.671550 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ebed944-88fa-4bbc-ad43-8dbf532e972e" containerName="maas-api" Jul 13 20:11:47.671686 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.671557 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="23576f99-61cb-41f2-9a35-89f53403830b" containerName="test-curl-7702-139685475591200" Jul 13 20:11:47.676057 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.676037 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:47.678178 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.678151 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dl6jf\"/\"openshift-service-ca.crt\"" Jul 13 20:11:47.678327 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.678188 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dl6jf\"/\"kube-root-ca.crt\"" Jul 13 20:11:47.678754 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.678725 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dl6jf\"/\"default-dockercfg-pfg74\"" Jul 13 20:11:47.685511 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.685481 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58"] Jul 13 20:11:47.737465 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.737410 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a8c4fbd2-d775-4ac0-a456-94f7254e64d4-podres\") pod \"perf-node-gather-daemonset-lwr58\" (UID: \"a8c4fbd2-d775-4ac0-a456-94f7254e64d4\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:47.737465 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.737465 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8c4fbd2-d775-4ac0-a456-94f7254e64d4-sys\") pod \"perf-node-gather-daemonset-lwr58\" (UID: \"a8c4fbd2-d775-4ac0-a456-94f7254e64d4\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:47.737726 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.737535 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8c4fbd2-d775-4ac0-a456-94f7254e64d4-lib-modules\") pod \"perf-node-gather-daemonset-lwr58\" (UID: \"a8c4fbd2-d775-4ac0-a456-94f7254e64d4\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:47.737726 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.737609 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjqf9\" (UniqueName: \"kubernetes.io/projected/a8c4fbd2-d775-4ac0-a456-94f7254e64d4-kube-api-access-fjqf9\") pod \"perf-node-gather-daemonset-lwr58\" (UID: \"a8c4fbd2-d775-4ac0-a456-94f7254e64d4\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:47.737726 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.737646 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a8c4fbd2-d775-4ac0-a456-94f7254e64d4-proc\") pod \"perf-node-gather-daemonset-lwr58\" (UID: \"a8c4fbd2-d775-4ac0-a456-94f7254e64d4\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:47.838399 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.838355 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a8c4fbd2-d775-4ac0-a456-94f7254e64d4-podres\") pod \"perf-node-gather-daemonset-lwr58\" (UID: \"a8c4fbd2-d775-4ac0-a456-94f7254e64d4\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:47.838399 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.838399 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8c4fbd2-d775-4ac0-a456-94f7254e64d4-sys\") pod \"perf-node-gather-daemonset-lwr58\" (UID: \"a8c4fbd2-d775-4ac0-a456-94f7254e64d4\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:47.838689 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.838476 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8c4fbd2-d775-4ac0-a456-94f7254e64d4-sys\") pod \"perf-node-gather-daemonset-lwr58\" (UID: \"a8c4fbd2-d775-4ac0-a456-94f7254e64d4\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:47.838689 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.838530 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a8c4fbd2-d775-4ac0-a456-94f7254e64d4-podres\") pod \"perf-node-gather-daemonset-lwr58\" (UID: \"a8c4fbd2-d775-4ac0-a456-94f7254e64d4\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:47.838689 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.838528 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8c4fbd2-d775-4ac0-a456-94f7254e64d4-lib-modules\") pod \"perf-node-gather-daemonset-lwr58\" (UID: \"a8c4fbd2-d775-4ac0-a456-94f7254e64d4\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:47.838689 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.838586 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjqf9\" (UniqueName: \"kubernetes.io/projected/a8c4fbd2-d775-4ac0-a456-94f7254e64d4-kube-api-access-fjqf9\") pod \"perf-node-gather-daemonset-lwr58\" (UID: \"a8c4fbd2-d775-4ac0-a456-94f7254e64d4\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:47.838689 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.838617 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a8c4fbd2-d775-4ac0-a456-94f7254e64d4-proc\") pod \"perf-node-gather-daemonset-lwr58\" (UID: \"a8c4fbd2-d775-4ac0-a456-94f7254e64d4\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:47.838689 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.838646 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8c4fbd2-d775-4ac0-a456-94f7254e64d4-lib-modules\") pod \"perf-node-gather-daemonset-lwr58\" (UID: \"a8c4fbd2-d775-4ac0-a456-94f7254e64d4\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:47.838985 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.838700 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a8c4fbd2-d775-4ac0-a456-94f7254e64d4-proc\") pod \"perf-node-gather-daemonset-lwr58\" (UID: \"a8c4fbd2-d775-4ac0-a456-94f7254e64d4\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:47.847892 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.847862 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjqf9\" (UniqueName: \"kubernetes.io/projected/a8c4fbd2-d775-4ac0-a456-94f7254e64d4-kube-api-access-fjqf9\") pod \"perf-node-gather-daemonset-lwr58\" (UID: \"a8c4fbd2-d775-4ac0-a456-94f7254e64d4\") " pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:47.988125 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:47.988011 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:48.135720 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:48.135692 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58"] Jul 13 20:11:48.138290 ip-10-0-132-20 kubenswrapper[2556]: W0713 20:11:48.138255 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda8c4fbd2_d775_4ac0_a456_94f7254e64d4.slice/crio-63ed496e112902aafe364f3b3b81ab0d18a38aedbfb0689bd4d4a623f68ba896 WatchSource:0}: Error finding container 63ed496e112902aafe364f3b3b81ab0d18a38aedbfb0689bd4d4a623f68ba896: Status 404 returned error can't find the container with id 63ed496e112902aafe364f3b3b81ab0d18a38aedbfb0689bd4d4a623f68ba896 Jul 13 20:11:48.639586 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:48.639559 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6575d7755d-kfhxr_0c4ae04c-364a-4660-8dea-f8adf53495d7/console/0.log" Jul 13 20:11:48.791030 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:48.790985 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" event={"ID":"a8c4fbd2-d775-4ac0-a456-94f7254e64d4","Type":"ContainerStarted","Data":"d6cd8428ffa11dfa9101d73f4661212c356a7644887c6f987615a44cbd4573fe"} Jul 13 20:11:48.791488 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:48.791043 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" event={"ID":"a8c4fbd2-d775-4ac0-a456-94f7254e64d4","Type":"ContainerStarted","Data":"63ed496e112902aafe364f3b3b81ab0d18a38aedbfb0689bd4d4a623f68ba896"} Jul 13 20:11:48.791488 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:48.791076 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:48.813532 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:48.813470 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" podStartSLOduration=1.8134523009999999 podStartE2EDuration="1.813452301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:11:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:11:48.810938648 +0000 UTC m=+3003.924846358" watchObservedRunningTime="2026-07-13 20:11:48.813452301 +0000 UTC m=+3003.927360011" Jul 13 20:11:50.454610 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:50.454564 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k24np_ee1b48b4-0511-4f05-8294-434364af55c1/dns/0.log" Jul 13 20:11:50.483533 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:50.483505 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k24np_ee1b48b4-0511-4f05-8294-434364af55c1/kube-rbac-proxy/0.log" Jul 13 20:11:50.625631 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:50.625599 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5fj4k_9b5b0a0b-15eb-4d6d-83cf-117a9009583b/dns-node-resolver/0.log" Jul 13 20:11:51.341651 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:51.341612 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5b6d67588f-zwg7r_af3abc1b-2644-468b-9144-d199a1417e34/registry/0.log" Jul 13 20:11:51.457557 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:51.457528 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rkbds_c0a858d7-aa1f-4af2-934a-240c6ed35046/node-ca/0.log" Jul 13 20:11:52.510085 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:52.510051 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mc4gc_a890d805-f6dc-483b-aaaa-8e6bfe48ef08/istio-proxy/0.log" Jul 13 20:11:52.835686 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:52.835652 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-l8mnm_16e74623-f867-449e-a739-4234322bb3e1/istio-proxy/0.log" Jul 13 20:11:52.862292 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:52.862257 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-l8mnm_16e74623-f867-449e-a739-4234322bb3e1/istio-proxy/1.log" Jul 13 20:11:53.774086 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:53.774058 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fplcj_23b8d70a-3533-428f-9b09-ad9c365fb03d/serve-healthcheck-canary/0.log" Jul 13 20:11:54.536589 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:54.536560 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r4flg_244c17d0-fb0f-4a75-a802-e99dd9dc3556/kube-rbac-proxy/0.log" Jul 13 20:11:54.558444 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:54.558416 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r4flg_244c17d0-fb0f-4a75-a802-e99dd9dc3556/exporter/0.log" Jul 13 20:11:54.581146 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:54.581115 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r4flg_244c17d0-fb0f-4a75-a802-e99dd9dc3556/extractor/0.log" Jul 13 20:11:54.807324 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:54.807230 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dl6jf/perf-node-gather-daemonset-lwr58" Jul 13 20:11:57.143469 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:57.143430 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-f5pvv_532f7bb2-91cc-4a13-9e97-7c379750744f/manager/0.log" Jul 13 20:11:57.344152 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:57.344125 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mc4gc_a890d805-f6dc-483b-aaaa-8e6bfe48ef08/istio-proxy/0.log" Jul 13 20:11:57.632698 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:57.632658 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-l8mnm_16e74623-f867-449e-a739-4234322bb3e1/istio-proxy/0.log" Jul 13 20:11:57.665298 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:57.665261 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-l8mnm_16e74623-f867-449e-a739-4234322bb3e1/istio-proxy/1.log" Jul 13 20:11:57.713701 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:57.713665 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6996d85cfb-j9wjq_de2480a1-3f58-4a81-835f-1ee281d2ba2e/manager/0.log" Jul 13 20:11:57.753607 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:57.753573 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6996d85cfb-j9wjq_de2480a1-3f58-4a81-835f-1ee281d2ba2e/copy-manifests/0.log" Jul 13 20:11:59.014645 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:59.014611 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mc4gc_a890d805-f6dc-483b-aaaa-8e6bfe48ef08/istio-proxy/0.log" Jul 13 20:11:59.027799 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:59.027759 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-655465cc8d-q5kzl_03b0ad2d-3e65-4b23-9a28-20ee30c350cf/manager/0.log" Jul 13 20:11:59.065279 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:59.065243 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-dc2nt_34b34ab1-edb9-4274-87be-0b2e174e2547/openshift-lws-operator/0.log" Jul 13 20:11:59.269757 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:59.269720 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-l8mnm_16e74623-f867-449e-a739-4234322bb3e1/istio-proxy/0.log" Jul 13 20:11:59.302617 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:11:59.302580 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-l8mnm_16e74623-f867-449e-a739-4234322bb3e1/istio-proxy/1.log" Jul 13 20:12:04.259441 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:12:04.259409 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-nrc2n_6a7d1437-bb2d-4b24-857c-7dde4b406770/migrator/0.log" Jul 13 20:12:04.345683 ip-10-0-132-20 kubenswrapper[2556]: I0713 20:12:04.345648 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-nrc2n_6a7d1437-bb2d-4b24-857c-7dde4b406770/graceful-termination/0.log"