Jul 16 04:39:07.139792 ip-10-0-132-245 systemd[1]: Starting Kubernetes Kubelet... Jul 16 04:39:07.726697 ip-10-0-132-245 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 04:39:07.726697 ip-10-0-132-245 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 04:39:07.726697 ip-10-0-132-245 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 04:39:07.726697 ip-10-0-132-245 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 04:39:07.726697 ip-10-0-132-245 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 04:39:07.726697 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.607226 2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 04:39:07.726697 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611211 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 04:39:07.726697 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611224 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 04:39:07.726697 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611228 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 04:39:07.726697 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611231 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 04:39:07.726697 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611235 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 04:39:07.726697 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611238 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 04:39:07.726697 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611241 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 04:39:07.726697 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611244 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611246 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611249 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611252 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611259 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611262 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611265 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611268 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611270 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611273 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611275 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611278 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611281 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611284 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611286 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611289 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611291 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611294 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611297 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611299 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 04:39:07.728712 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611302 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 04:39:07.682571 ip-10-0-132-245 systemd[1]: Started Kubernetes Kubelet. Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611305 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611309 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611312 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611316 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611319 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611322 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611325 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611328 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611331 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611333 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611336 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611339 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611342 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611345 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611348 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611351 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611354 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611356 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 04:39:07.747828 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611359 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611362 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611364 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611366 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611369 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611372 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611374 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611377 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611379 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611382 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611384 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611387 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611390 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611392 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611396 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611400 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611403 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611406 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611409 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611412 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 04:39:07.761651 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611414 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611417 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611420 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611423 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611425 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611428 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611431 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611433 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611435 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611438 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611442 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611444 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611447 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611450 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611453 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611455 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611458 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611460 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611463 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611465 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 04:39:07.765276 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611468 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611470 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611472 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611475 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611477 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611480 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611482 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611485 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611488 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611490 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611493 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611495 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.611498 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617245 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617256 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617259 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617262 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617266 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617269 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617272 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 04:39:07.954792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617276 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617279 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617282 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617285 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617288 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617291 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617294 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617296 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617299 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617302 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617304 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617307 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617309 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617312 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617314 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617317 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617320 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617322 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617325 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617328 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 04:39:07.956214 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617330 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617333 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617335 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617338 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617340 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617343 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617345 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617347 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617350 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617353 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617356 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617359 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617361 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617364 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617368 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617371 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617374 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617377 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617380 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 04:39:07.957376 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617383 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617385 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617388 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617390 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617393 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617396 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617398 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617402 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617405 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617408 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617410 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617413 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617415 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617418 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617420 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617423 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617425 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617428 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617431 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617434 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 04:39:07.959237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617436 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617439 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617442 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617444 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617447 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617450 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617453 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617455 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617457 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617460 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617463 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617466 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617468 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617471 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617474 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617478 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617480 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617484 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617486 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 04:39:07.960324 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617488 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617491 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617493 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617496 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617498 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617518 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617521 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617524 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617527 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617529 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617532 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617535 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617539 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617542 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617898 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617904 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617906 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617910 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617914 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 04:39:07.961193 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617917 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617921 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617925 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617929 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617932 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617936 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617938 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617941 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617944 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617946 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617949 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617952 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617954 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617957 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617960 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617962 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617965 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617967 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617970 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 04:39:07.963328 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617972 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617975 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617977 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617980 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617982 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617985 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617987 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617990 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617992 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617995 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.617998 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618003 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618006 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618009 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618011 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618014 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618016 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618018 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618021 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618023 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 04:39:07.964397 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618026 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618028 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618031 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618033 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618036 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618038 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618041 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618043 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618046 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618048 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618051 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618054 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618056 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618059 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618061 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618063 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618066 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618069 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618072 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618075 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 04:39:07.966291 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618077 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618079 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618082 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618089 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618092 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618095 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618097 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618100 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618102 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618105 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618107 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618110 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618112 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618115 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618118 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618120 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618122 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618125 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618128 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 04:39:07.967373 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618130 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618133 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618135 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618138 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618140 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618143 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618145 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618148 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618151 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618155 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618158 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618160 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618163 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618166 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618170 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618173 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618248 2581 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618260 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618269 2581 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618275 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618281 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 04:39:07.969058 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618286 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618290 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618294 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618298 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618301 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618305 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618308 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618311 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618314 2581 flags.go:64] FLAG: --cgroup-root="" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618316 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618319 2581 flags.go:64] FLAG: --client-ca-file="" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618323 2581 flags.go:64] FLAG: --cloud-provider="external" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618326 2581 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618330 2581 flags.go:64] FLAG: --cluster-domain="" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618333 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618336 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618339 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618342 2581 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618350 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618353 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618359 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618362 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618365 2581 flags.go:64] FLAG: --contention-profiling="false" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618368 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 04:39:07.970136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618371 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618374 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618377 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618383 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618386 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618389 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618391 2581 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618395 2581 flags.go:64] FLAG: --enable-server="true" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618398 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618402 2581 flags.go:64] FLAG: --event-burst="100" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618405 2581 flags.go:64] FLAG: --event-qps="50" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618408 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618411 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618414 2581 flags.go:64] FLAG: --eviction-hard="" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618417 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618420 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618423 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618426 2581 flags.go:64] FLAG: --eviction-soft="" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618429 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618432 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618435 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618438 2581 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618441 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618444 2581 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618446 2581 flags.go:64] FLAG: --feature-gates="" Jul 16 04:39:07.971266 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618450 2581 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618453 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618456 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618460 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618463 2581 flags.go:64] FLAG: --healthz-port="10248" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618466 2581 flags.go:64] FLAG: --help="false" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618469 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-132-245.ec2.internal" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618472 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618475 2581 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618478 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618482 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618485 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618488 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618492 2581 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618495 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618497 2581 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618528 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618533 2581 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618536 2581 flags.go:64] FLAG: --kube-reserved="" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618539 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618542 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618545 2581 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618549 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618551 2581 flags.go:64] FLAG: --lock-file="" Jul 16 04:39:07.972606 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618554 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618557 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618560 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618565 2581 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618569 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618573 2581 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618578 2581 flags.go:64] FLAG: --logging-format="text" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618582 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618587 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618592 2581 flags.go:64] FLAG: --manifest-url="" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618596 2581 flags.go:64] FLAG: --manifest-url-header="" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618602 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618605 2581 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618609 2581 flags.go:64] FLAG: --max-pods="110" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618612 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618615 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618618 2581 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618621 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618625 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618628 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618631 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618638 2581 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618641 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618644 2581 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 04:39:07.973727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618647 2581 flags.go:64] FLAG: --pod-cidr="" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618650 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618656 2581 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618659 2581 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618662 2581 flags.go:64] FLAG: --pods-per-core="0" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618664 2581 flags.go:64] FLAG: --port="10250" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618668 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618670 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d0765e5ebfff32a1" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618673 2581 flags.go:64] FLAG: --qos-reserved="" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618676 2581 flags.go:64] FLAG: --read-only-port="10255" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618679 2581 flags.go:64] FLAG: --register-node="true" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618682 2581 flags.go:64] FLAG: --register-with-taints="" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618686 2581 flags.go:64] FLAG: --registry-burst="10" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618690 2581 flags.go:64] FLAG: --registry-qps="5" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618692 2581 flags.go:64] FLAG: --reserved-cpus="" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618695 2581 flags.go:64] FLAG: --reserved-memory="" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618699 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618702 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618705 2581 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618707 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618711 2581 flags.go:64] FLAG: --runonce="false" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618714 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618718 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618721 2581 flags.go:64] FLAG: --seccomp-default="false" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618724 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 04:39:07.974835 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618727 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618731 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618734 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618737 2581 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618740 2581 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618743 2581 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618746 2581 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618749 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618752 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618755 2581 flags.go:64] FLAG: --system-cgroups="" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618757 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618763 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618765 2581 flags.go:64] FLAG: --tls-cert-file="" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618768 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618773 2581 flags.go:64] FLAG: --tls-min-version="" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618775 2581 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618778 2581 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618781 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618784 2581 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618787 2581 flags.go:64] FLAG: --v="2" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618790 2581 flags.go:64] FLAG: --version="false" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618795 2581 flags.go:64] FLAG: --vmodule="" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618802 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.618805 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618914 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 04:39:07.975944 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618917 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618921 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618925 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618928 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618931 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618934 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618937 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618939 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618943 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618946 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618948 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618951 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618954 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618956 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618959 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618961 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618964 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618967 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618969 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618972 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 04:39:07.977048 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618974 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618978 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618981 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618984 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618988 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618991 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618994 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618996 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.618999 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619004 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619007 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619009 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619012 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619015 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619019 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619021 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619024 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619027 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619029 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619032 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 04:39:07.977916 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619035 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619038 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619041 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619043 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619046 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619048 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619051 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619054 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619056 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619059 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619063 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619067 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619069 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619072 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619075 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619077 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619081 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619083 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619086 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 04:39:07.978959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619088 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619090 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619093 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619096 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619099 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619101 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619104 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619108 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619111 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619113 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619116 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619118 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619121 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619125 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619128 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619130 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619133 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619135 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619138 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619140 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 04:39:07.979821 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619143 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 04:39:07.980695 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619145 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 04:39:07.980695 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619148 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 04:39:07.980695 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619151 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 04:39:07.980695 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619153 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 04:39:07.980695 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619156 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 04:39:07.980695 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619158 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 04:39:07.980695 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619161 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 04:39:07.980695 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619164 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 04:39:07.980695 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619166 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 04:39:07.980695 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619169 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 04:39:07.980695 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619171 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 04:39:07.980695 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619174 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 04:39:07.980695 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619178 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 04:39:07.980695 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619181 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 04:39:07.980695 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619184 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 04:39:07.980695 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619188 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 04:39:07.980695 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619190 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 04:39:07.980695 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.619193 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 04:39:07.982372 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.619835 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 04:39:07.982372 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.627749 2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 04:39:07.982372 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.627764 2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 04:39:07.982372 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.627777 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 04:39:07.982372 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.627783 2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 04:39:07.982372 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627840 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 04:39:07.982372 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627846 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 04:39:07.982372 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627851 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 04:39:07.982372 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627855 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 04:39:07.982372 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627858 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 04:39:07.982372 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627861 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 04:39:07.982372 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627864 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 04:39:07.982372 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627866 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 04:39:07.982372 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627870 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 04:39:07.982372 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627872 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627875 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627878 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627881 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627883 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627887 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627891 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627894 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627896 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627902 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627905 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627907 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627910 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627912 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627915 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627918 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627921 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627923 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627926 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627928 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 04:39:07.983219 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627931 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627934 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627936 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627939 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627942 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627944 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627949 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627951 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627954 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627956 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627959 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627962 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627964 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627967 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627969 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627971 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627974 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627976 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627979 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627981 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 04:39:07.984942 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627984 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627988 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627991 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627994 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627996 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.627999 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628001 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628004 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628007 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628009 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628012 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628014 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628017 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628020 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628022 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628025 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628027 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628030 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628034 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 04:39:07.985808 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628036 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628039 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628042 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628044 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628047 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628049 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628052 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628054 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628057 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628059 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628062 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628064 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628067 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628071 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628075 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628078 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628081 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628084 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628087 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 04:39:07.987088 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628089 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 04:39:07.987951 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628092 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 04:39:07.987951 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628095 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 04:39:07.987951 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628097 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 04:39:07.987951 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628100 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 04:39:07.987951 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628103 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 04:39:07.987951 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628105 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 04:39:07.987951 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628108 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 04:39:07.987951 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628111 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 04:39:07.987951 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628113 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 04:39:07.987951 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628116 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 04:39:07.987951 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628118 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 04:39:07.987951 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.628125 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 04:39:07.987951 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628232 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 04:39:07.987951 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628236 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 04:39:07.987951 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628239 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628242 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628245 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628248 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628250 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628253 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628256 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628258 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628261 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628264 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628266 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628269 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628271 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628275 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628278 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628281 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628283 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628286 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628288 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628291 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 04:39:07.988625 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628293 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628296 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628299 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628301 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628304 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628306 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628309 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628311 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628314 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628317 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628320 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628323 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628325 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628328 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628330 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628334 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628338 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628340 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628343 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628354 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 04:39:07.989531 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628357 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628360 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628364 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628367 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628371 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628375 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628377 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628380 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628383 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628386 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628389 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628391 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628394 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628396 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628399 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628401 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628404 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628406 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628409 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 04:39:07.990419 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628412 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628414 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628419 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628421 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628424 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628426 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628429 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628433 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628436 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628439 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628441 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628444 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628447 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628449 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628452 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628454 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628457 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628459 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628464 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 04:39:07.991927 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628467 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 04:39:07.992781 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628469 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 04:39:07.992781 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628472 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 04:39:07.992781 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628474 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 04:39:07.992781 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628477 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 04:39:07.992781 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628479 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 04:39:07.992781 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628482 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 04:39:07.992781 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628484 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 04:39:07.992781 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628487 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 04:39:07.992781 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628489 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 04:39:07.992781 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628492 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 04:39:07.992781 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628494 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 04:39:07.992781 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628497 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 04:39:07.992781 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628513 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 04:39:07.992781 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628516 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 04:39:07.992781 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628520 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 04:39:07.992781 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628523 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 04:39:07.992781 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628526 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 04:39:07.992781 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:07.628529 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 04:39:07.993866 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.628534 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 04:39:07.993866 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.628631 2581 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 04:39:07.993866 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.630794 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 04:39:07.993866 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.632371 2581 server.go:1019] "Starting client certificate rotation" Jul 16 04:39:07.993866 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.632467 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 04:39:07.993866 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.632551 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 04:39:07.993866 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.635200 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 04:39:07.993866 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.638242 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 04:39:07.993866 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.647989 2581 log.go:25] "Validated CRI v1 runtime API" Jul 16 04:39:07.993866 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.650329 2581 log.go:25] "Validated CRI v1 image API" Jul 16 04:39:07.993866 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.651875 2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 04:39:07.993866 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.654438 2581 fs.go:135] Filesystem UUIDs: map[0dabd16f-53bb-44fe-ab70-968234cf8958:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 abb6745a-5f4d-46ca-a20e-87fdfc46704e:/dev/nvme0n1p3] Jul 16 04:39:07.994491 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.654452 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 04:39:08.161331 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.659313 2581 manager.go:216] Machine: {Timestamp:2026-07-16 04:39:07.658857485 +0000 UTC m=+0.384325998 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3115096 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26088d5af42308cc91060a4c763d8c SystemUUID:ec26088d-5af4-2308-cc91-060a4c763d8c BootID:49258cc8-4ed0-46f0-813b-12e7e1d647a5 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cc:8e:8f:b6:f9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cc:8e:8f:b6:f9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:71:ab:c2:36:85 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 04:39:08.161331 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.659409 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 04:39:08.161331 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.659475 2581 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 04:39:08.161331 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.662511 2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 04:39:08.161331 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.662534 2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-245.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 04:39:08.161331 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.663278 2581 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 04:39:08.161331 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.663288 2581 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 04:39:08.161331 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.663295 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.664222 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.665917 2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.671017 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.671152 2581 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.671845 2581 kubelet.go:486] "Attempting to sync node with API server" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.671857 2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.671871 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.671881 2581 kubelet.go:398] "Adding apiserver pod source" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.671889 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.672213 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.672228 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.674425 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.674625 2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.676917 2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.676944 2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.677088 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.677100 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.677106 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.677111 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.677119 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.677124 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.677804 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.677811 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.677819 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.677825 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.677833 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.677840 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.678573 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.678580 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.681786 2581 server.go:1268] "Started kubelet" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.681945 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.682155 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.682191 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.683671 2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:07.686952 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-245.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:07.687196 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.687329 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-245.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.688198 2581 server.go:310] "Adding debug handlers to kubelet server" Jul 16 04:39:08.161774 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:07.688421 2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:07.689536 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-245.ec2.internal.18c2ab4c8e130516 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-245.ec2.internal,UID:ip-10-0-132-245.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-245.ec2.internal,},FirstTimestamp:2026-07-16 04:39:07.681613078 +0000 UTC m=+0.407081592,LastTimestamp:2026-07-16 04:39:07.681613078 +0000 UTC m=+0.407081592,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-245.ec2.internal,}" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.694002 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.694006 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:07.694082 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.694720 2581 factory.go:55] Registering systemd factory Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.694738 2581 factory.go:223] Registration of the systemd container factory successfully Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.694739 2581 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.694745 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.694754 2581 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.694843 2581 reconstruct.go:97] "Volume reconstruction finished" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.694853 2581 reconciler.go:29] "Reconciler: start to sync state" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.694927 2581 factory.go:153] Registering CRI-O factory Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.694941 2581 factory.go:223] Registration of the crio container factory successfully Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.694993 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.695013 2581 factory.go:103] Registering Raw factory Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.695029 2581 manager.go:1196] Started watching for new ooms in manager Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.695313 2581 manager.go:349] Starting recovery of all containers Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:07.696223 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:07.696316 2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-245.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.703466 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rp2tz" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.706490 2581 manager.go:354] Recovery completed Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.710936 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.712447 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rp2tz" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.714998 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.715026 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.715040 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.715614 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.715618 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.715674 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.715695 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.716338 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:39:08.162863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.716365 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.716389 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.716567 2581 policy_none.go:49] "None policy: Start" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.716582 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.716593 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.717449 2581 policy_none.go:47] "Start" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:07.794642 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.809024 2581 manager.go:337] "Starting Device Plugin manager" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:07.809050 2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.809059 2581 server.go:87] "Starting device plugin registration server" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.809243 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.809255 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.809348 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.809428 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.809437 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:07.809844 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:07.809886 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-245.ec2.internal\" not found" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.831306 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.832428 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.832458 2581 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.832478 2581 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:07.832522 2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.835610 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:07.903713 2581 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-245.ec2.internal\" not found" node="ip-10-0-132-245.ec2.internal" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.909822 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.910690 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.910720 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.910735 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.910763 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-245.ec2.internal" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.920261 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-245.ec2.internal" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:07.920280 2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-245.ec2.internal\": node \"ip-10-0-132-245.ec2.internal\" not found" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.933162 2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal"] Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.933216 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.933942 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.933969 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.933980 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.936127 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.936221 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal" Jul 16 04:39:08.163727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.936249 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.936759 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.936773 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.936783 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.936808 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.936828 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.936848 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.938834 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.938856 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.939453 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.939478 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.939488 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:07.953597 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.996343 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/40de15c40dab007c9ee75dd5f90e7e7c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-245.ec2.internal\" (UID: \"40de15c40dab007c9ee75dd5f90e7e7c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.996372 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0991fafe223622889a10e1583a4386ad-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal\" (UID: \"0991fafe223622889a10e1583a4386ad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:07.996390 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0991fafe223622889a10e1583a4386ad-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal\" (UID: \"0991fafe223622889a10e1583a4386ad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:08.002144 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-245.ec2.internal\" not found" node="ip-10-0-132-245.ec2.internal" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:08.008864 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-245.ec2.internal\" not found" node="ip-10-0-132-245.ec2.internal" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:08.054494 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:08.096574 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0991fafe223622889a10e1583a4386ad-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal\" (UID: \"0991fafe223622889a10e1583a4386ad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:08.096533 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0991fafe223622889a10e1583a4386ad-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal\" (UID: \"0991fafe223622889a10e1583a4386ad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal" Jul 16 04:39:08.164818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:08.096638 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0991fafe223622889a10e1583a4386ad-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal\" (UID: \"0991fafe223622889a10e1583a4386ad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal" Jul 16 04:39:08.165420 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:08.096658 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/40de15c40dab007c9ee75dd5f90e7e7c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-245.ec2.internal\" (UID: \"40de15c40dab007c9ee75dd5f90e7e7c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal" Jul 16 04:39:08.165420 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:08.096691 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/40de15c40dab007c9ee75dd5f90e7e7c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-245.ec2.internal\" (UID: \"40de15c40dab007c9ee75dd5f90e7e7c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal" Jul 16 04:39:08.165420 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:08.096732 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0991fafe223622889a10e1583a4386ad-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal\" (UID: \"0991fafe223622889a10e1583a4386ad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal" Jul 16 04:39:08.165420 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:08.155007 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found" Jul 16 04:39:08.338724 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:08.255666 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found" Jul 16 04:39:08.338724 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:08.304087 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal" Jul 16 04:39:08.338724 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:08.311634 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal" Jul 16 04:39:08.516093 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:08.356600 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found" Jul 16 04:39:08.516093 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:08.456940 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found" Jul 16 04:39:08.650986 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:08.557057 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found" Jul 16 04:39:08.650986 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:08.631893 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 04:39:08.650986 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:08.632021 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 04:39:08.660530 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:08.658064 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found" Jul 16 04:39:09.164646 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:09.164551 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found" Jul 16 04:39:09.164646 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.164570 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 04:34:07 +0000 UTC" deadline="2027-12-16 04:30:07.331996314 +0000 UTC" Jul 16 04:39:09.164646 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.164607 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12431h50m58.167394474s" Jul 16 04:39:09.167375 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.167356 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 04:39:09.167496 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.167480 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 04:39:09.171432 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.171413 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 04:39:09.194272 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.194242 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal" Jul 16 04:39:09.205881 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.205863 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 04:39:09.206891 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.206878 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal" Jul 16 04:39:09.219575 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.219554 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 04:39:09.352237 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:09.352180 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0991fafe223622889a10e1583a4386ad.slice/crio-ad9fac6db4a0a6f08bd330f1a0df3426da1d68fe1adde20f0dd37a9417742190 WatchSource:0}: Error finding container ad9fac6db4a0a6f08bd330f1a0df3426da1d68fe1adde20f0dd37a9417742190: Status 404 returned error can't find the container with id ad9fac6db4a0a6f08bd330f1a0df3426da1d68fe1adde20f0dd37a9417742190 Jul 16 04:39:09.352489 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:09.352475 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40de15c40dab007c9ee75dd5f90e7e7c.slice/crio-14e0694b5e816f9cc02a4f155c6b71c3ad23d931e5d25629d7dfd8bab5661205 WatchSource:0}: Error finding container 14e0694b5e816f9cc02a4f155c6b71c3ad23d931e5d25629d7dfd8bab5661205: Status 404 returned error can't find the container with id 14e0694b5e816f9cc02a4f155c6b71c3ad23d931e5d25629d7dfd8bab5661205 Jul 16 04:39:09.357855 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.357838 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 04:39:09.496142 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.496063 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 04:39:09.674251 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.674216 2581 apiserver.go:52] "Watching apiserver" Jul 16 04:39:09.681844 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.681821 2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 04:39:09.684443 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.683302 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal","openshift-multus/multus-additional-cni-plugins-m9m2p","openshift-multus/multus-hnbvz","openshift-network-operator/iptables-alerter-47pt2","kube-system/konnectivity-agent-fnkvt","kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx","openshift-cluster-node-tuning-operator/tuned-lplz2","openshift-multus/network-metrics-daemon-7g5n5","openshift-network-diagnostics/network-check-target-bwl6r","openshift-ovn-kubernetes/ovnkube-node-lrbbv","kube-system/global-pull-secret-syncer-g6vzk","openshift-dns/node-resolver-xjxgr","openshift-image-registry/node-ca-cngtn"] Jul 16 04:39:09.687719 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.687701 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.690105 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.690085 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xjxgr" Jul 16 04:39:09.690546 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.690529 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 04:39:09.691109 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.691091 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-s2s5z\"" Jul 16 04:39:09.691184 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.691124 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 04:39:09.691505 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.691489 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 04:39:09.692279 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.692255 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-v8sr4\"" Jul 16 04:39:09.692432 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.692418 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.692855 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.692835 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 04:39:09.693119 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.693101 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 04:39:09.694189 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.694174 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 04:39:09.694638 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.694608 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lcn5n\"" Jul 16 04:39:09.694749 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.694733 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 04:39:09.694857 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.694837 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-47pt2" Jul 16 04:39:09.694970 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.694952 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 04:39:09.695257 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.695238 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 04:39:09.695367 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.695348 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 04:39:09.697315 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.697296 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fnkvt" Jul 16 04:39:09.699452 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.699434 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 04:39:09.699757 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.699738 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cngtn" Jul 16 04:39:09.699842 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.699811 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 04:39:09.699907 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.699862 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 04:39:09.700615 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.700596 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-w926s\"" Jul 16 04:39:09.700702 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.700685 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-t7bbd\"" Jul 16 04:39:09.701218 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.701202 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 04:39:09.702092 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.702066 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:09.702189 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:09.702147 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7g5n5" podUID="2f51d802-9618-4e88-8fc8-78b2a3788356" Jul 16 04:39:09.702411 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.702395 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 04:39:09.703648 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.703630 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-45lbx\"" Jul 16 04:39:09.703775 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.703649 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 04:39:09.703849 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.703822 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 04:39:09.704143 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.704114 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 04:39:09.704364 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.704350 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.707001 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.706982 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.707305 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.707283 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 04:39:09.707603 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.707575 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 04:39:09.707704 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.707687 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dltjz\"" Jul 16 04:39:09.709219 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.709199 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 04:39:09.709832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.709816 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 04:39:09.710097 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.710081 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:09.710286 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:09.710266 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwl6r" podUID="7c288293-bb92-454b-b0de-91d7317cda4b" Jul 16 04:39:09.710685 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.710658 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 04:39:09.710786 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.710773 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-62bdd\"" Jul 16 04:39:09.713002 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.712984 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.716006 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.715984 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:09.716113 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:09.716034 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6vzk" podUID="5da9fe69-0b36-4198-a5ff-329ddaee7389" Jul 16 04:39:09.716217 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.716202 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 04:39:09.716408 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.716391 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 04:39:09.716462 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.716408 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 04:39:09.716750 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.716733 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 04:39:09.717590 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.717563 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 04:39:09.717721 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.717707 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 04:39:09.717797 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.717757 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hd96j\"" Jul 16 04:39:09.778092 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778060 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-registration-dir\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.778196 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778096 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-system-cni-dir\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.778196 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778114 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-os-release\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.778196 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778129 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-host-var-lib-cni-bin\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.778196 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778181 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3b736bea-a20b-4220-b592-3819366c1ebd-konnectivity-ca\") pod \"konnectivity-agent-fnkvt\" (UID: \"3b736bea-a20b-4220-b592-3819366c1ebd\") " pod="kube-system/konnectivity-agent-fnkvt" Jul 16 04:39:09.778325 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778211 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-device-dir\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.778325 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778230 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-cnibin\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.778325 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778246 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-hostroot\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.778325 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778261 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhpgj\" (UniqueName: \"kubernetes.io/projected/0d695a61-7e54-42ac-9251-a3d84eeb3baa-kube-api-access-bhpgj\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.778325 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778280 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d695a61-7e54-42ac-9251-a3d84eeb3baa-cni-binary-copy\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.778325 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778295 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd-host-slash\") pod \"iptables-alerter-47pt2\" (UID: \"e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd\") " pod="openshift-network-operator/iptables-alerter-47pt2" Jul 16 04:39:09.778325 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778309 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3b736bea-a20b-4220-b592-3819366c1ebd-agent-certs\") pod \"konnectivity-agent-fnkvt\" (UID: \"3b736bea-a20b-4220-b592-3819366c1ebd\") " pod="kube-system/konnectivity-agent-fnkvt" Jul 16 04:39:09.778553 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778335 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-etc-selinux\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.778553 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778361 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0d695a61-7e54-42ac-9251-a3d84eeb3baa-multus-daemon-config\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.778553 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778378 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rt7z\" (UniqueName: \"kubernetes.io/projected/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-kube-api-access-9rt7z\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.778553 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778394 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgbnh\" (UniqueName: \"kubernetes.io/projected/01e9bde7-f187-4fe4-b933-e31dfc8de92b-kube-api-access-xgbnh\") pod \"node-resolver-xjxgr\" (UID: \"01e9bde7-f187-4fe4-b933-e31dfc8de92b\") " pod="openshift-dns/node-resolver-xjxgr" Jul 16 04:39:09.778553 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778410 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-multus-cni-dir\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.778553 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778444 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-host-run-k8s-cni-cncf-io\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.778553 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778480 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rttq\" (UniqueName: \"kubernetes.io/projected/e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd-kube-api-access-5rttq\") pod \"iptables-alerter-47pt2\" (UID: \"e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd\") " pod="openshift-network-operator/iptables-alerter-47pt2" Jul 16 04:39:09.778553 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778529 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01e9bde7-f187-4fe4-b933-e31dfc8de92b-tmp-dir\") pod \"node-resolver-xjxgr\" (UID: \"01e9bde7-f187-4fe4-b933-e31dfc8de92b\") " pod="openshift-dns/node-resolver-xjxgr" Jul 16 04:39:09.778553 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778553 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-socket-dir\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.778824 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778568 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-host-run-netns\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.778824 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778583 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-host-var-lib-kubelet\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.778824 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778600 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-multus-conf-dir\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.778824 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778628 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-host-run-multus-certs\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.778824 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778646 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.778824 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778661 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-multus-socket-dir-parent\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.778824 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778675 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-host-var-lib-cni-multus\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.778824 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778719 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-etc-kubernetes\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.778824 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778739 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd-iptables-alerter-script\") pod \"iptables-alerter-47pt2\" (UID: \"e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd\") " pod="openshift-network-operator/iptables-alerter-47pt2" Jul 16 04:39:09.778824 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778755 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-sys-fs\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.778824 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.778776 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/01e9bde7-f187-4fe4-b933-e31dfc8de92b-hosts-file\") pod \"node-resolver-xjxgr\" (UID: \"01e9bde7-f187-4fe4-b933-e31dfc8de92b\") " pod="openshift-dns/node-resolver-xjxgr" Jul 16 04:39:09.795863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.795842 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 04:39:09.801997 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.801982 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4z9xj" Jul 16 04:39:09.813778 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.813759 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4z9xj" Jul 16 04:39:09.879521 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.879493 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-cnibin\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.879647 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.879544 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-hostroot\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.879647 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.879595 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-hostroot\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.879647 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.879619 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-cnibin\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.879816 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.879658 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhpgj\" (UniqueName: \"kubernetes.io/projected/0d695a61-7e54-42ac-9251-a3d84eeb3baa-kube-api-access-bhpgj\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.879816 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.879702 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpjdr\" (UniqueName: \"kubernetes.io/projected/852b5867-4340-4c84-9da8-8e4f02270f0d-kube-api-access-rpjdr\") pod \"node-ca-cngtn\" (UID: \"852b5867-4340-4c84-9da8-8e4f02270f0d\") " pod="openshift-image-registry/node-ca-cngtn" Jul 16 04:39:09.879816 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.879742 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4c5z\" (UniqueName: \"kubernetes.io/projected/2f51d802-9618-4e88-8fc8-78b2a3788356-kube-api-access-q4c5z\") pod \"network-metrics-daemon-7g5n5\" (UID: \"2f51d802-9618-4e88-8fc8-78b2a3788356\") " pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:09.879816 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.879769 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret\") pod \"global-pull-secret-syncer-g6vzk\" (UID: \"5da9fe69-0b36-4198-a5ff-329ddaee7389\") " pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:09.879816 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.879804 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-cni-bin\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.880011 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.879829 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/852b5867-4340-4c84-9da8-8e4f02270f0d-host\") pod \"node-ca-cngtn\" (UID: \"852b5867-4340-4c84-9da8-8e4f02270f0d\") " pod="openshift-image-registry/node-ca-cngtn" Jul 16 04:39:09.880011 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.879851 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/852b5867-4340-4c84-9da8-8e4f02270f0d-serviceca\") pod \"node-ca-cngtn\" (UID: \"852b5867-4340-4c84-9da8-8e4f02270f0d\") " pod="openshift-image-registry/node-ca-cngtn" Jul 16 04:39:09.880011 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.879874 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99e2e193-30b8-43ed-9e81-023f10d73b10-system-cni-dir\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.880011 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.879902 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/99e2e193-30b8-43ed-9e81-023f10d73b10-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.880011 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.879926 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-tmp\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.880011 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.879964 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/95a3a732-50cf-47f5-9629-0b3d241d017a-ovnkube-script-lib\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.880261 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880044 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgbnh\" (UniqueName: \"kubernetes.io/projected/01e9bde7-f187-4fe4-b933-e31dfc8de92b-kube-api-access-xgbnh\") pod \"node-resolver-xjxgr\" (UID: \"01e9bde7-f187-4fe4-b933-e31dfc8de92b\") " pod="openshift-dns/node-resolver-xjxgr" Jul 16 04:39:09.880261 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880082 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-run-systemd\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.880261 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880114 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-host-run-k8s-cni-cncf-io\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.880261 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880140 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-var-lib-openvswitch\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.880261 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880169 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-run-ovn-kubernetes\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.880261 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880190 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-host-run-k8s-cni-cncf-io\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.880261 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880220 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/95a3a732-50cf-47f5-9629-0b3d241d017a-ovnkube-config\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.880261 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880259 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-host-run-netns\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.880639 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880285 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-multus-conf-dir\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.880639 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880311 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-host-run-multus-certs\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.880639 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880336 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.880639 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880335 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-host-run-netns\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.880639 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880350 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-multus-conf-dir\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.880639 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880375 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-host-run-multus-certs\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.880639 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880409 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-sysconfig\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.880639 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880446 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-kubelet\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.880639 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880474 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-log-socket\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.880639 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880413 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.880639 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880503 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-multus-socket-dir-parent\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.880639 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880557 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-etc-kubernetes\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.881153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880674 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-multus-socket-dir-parent\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.881153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880665 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-sys-fs\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.881153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880728 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-etc-kubernetes\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.881153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880772 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trjwk\" (UniqueName: \"kubernetes.io/projected/99e2e193-30b8-43ed-9e81-023f10d73b10-kube-api-access-trjwk\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.881153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880827 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-sys-fs\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.881153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880840 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-sysctl-conf\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.881153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880868 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-slash\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.881153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880890 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/01e9bde7-f187-4fe4-b933-e31dfc8de92b-hosts-file\") pod \"node-resolver-xjxgr\" (UID: \"01e9bde7-f187-4fe4-b933-e31dfc8de92b\") " pod="openshift-dns/node-resolver-xjxgr" Jul 16 04:39:09.881153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880917 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-run-openvswitch\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.881153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880944 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-system-cni-dir\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.881153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880970 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-os-release\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.881153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880982 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/01e9bde7-f187-4fe4-b933-e31dfc8de92b-hosts-file\") pod \"node-resolver-xjxgr\" (UID: \"01e9bde7-f187-4fe4-b933-e31dfc8de92b\") " pod="openshift-dns/node-resolver-xjxgr" Jul 16 04:39:09.881153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.880994 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-host-var-lib-cni-bin\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.881153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881020 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-device-dir\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.881153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881038 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-system-cni-dir\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.881963 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881047 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/99e2e193-30b8-43ed-9e81-023f10d73b10-os-release\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.881963 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881072 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-run\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.881963 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881093 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-host-var-lib-cni-bin\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.881963 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881098 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99e2e193-30b8-43ed-9e81-023f10d73b10-cni-binary-copy\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.881963 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881122 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-sys\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.881963 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881148 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-var-lib-kubelet\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.881963 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881155 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-os-release\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.881963 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881172 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-run-netns\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.881963 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881201 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-etc-openvswitch\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.881963 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881212 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-device-dir\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.881963 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881227 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/95a3a732-50cf-47f5-9629-0b3d241d017a-ovn-node-metrics-cert\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.881963 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881254 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d695a61-7e54-42ac-9251-a3d84eeb3baa-cni-binary-copy\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.881963 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881292 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd-host-slash\") pod \"iptables-alerter-47pt2\" (UID: \"e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd\") " pod="openshift-network-operator/iptables-alerter-47pt2" Jul 16 04:39:09.881963 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881370 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd-host-slash\") pod \"iptables-alerter-47pt2\" (UID: \"e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd\") " pod="openshift-network-operator/iptables-alerter-47pt2" Jul 16 04:39:09.882675 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881465 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3b736bea-a20b-4220-b592-3819366c1ebd-agent-certs\") pod \"konnectivity-agent-fnkvt\" (UID: \"3b736bea-a20b-4220-b592-3819366c1ebd\") " pod="kube-system/konnectivity-agent-fnkvt" Jul 16 04:39:09.882675 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881503 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-etc-selinux\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.882675 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881549 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5da9fe69-0b36-4198-a5ff-329ddaee7389-dbus\") pod \"global-pull-secret-syncer-g6vzk\" (UID: \"5da9fe69-0b36-4198-a5ff-329ddaee7389\") " pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:09.882675 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881581 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-modprobe-d\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.882675 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881607 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-systemd-units\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.882675 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881631 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-cni-netd\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.882675 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881660 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0d695a61-7e54-42ac-9251-a3d84eeb3baa-multus-daemon-config\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.882675 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881687 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-etc-selinux\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.882675 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881687 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rt7z\" (UniqueName: \"kubernetes.io/projected/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-kube-api-access-9rt7z\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.882675 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881724 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/99e2e193-30b8-43ed-9e81-023f10d73b10-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.882675 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.881833 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 04:39:09.882675 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.882455 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d695a61-7e54-42ac-9251-a3d84eeb3baa-cni-binary-copy\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.882675 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.882490 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0d695a61-7e54-42ac-9251-a3d84eeb3baa-multus-daemon-config\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.883263 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.882719 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-sysctl-d\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.883263 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.882770 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-multus-cni-dir\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.883263 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.882800 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rttq\" (UniqueName: \"kubernetes.io/projected/e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd-kube-api-access-5rttq\") pod \"iptables-alerter-47pt2\" (UID: \"e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd\") " pod="openshift-network-operator/iptables-alerter-47pt2" Jul 16 04:39:09.883263 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.882826 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01e9bde7-f187-4fe4-b933-e31dfc8de92b-tmp-dir\") pod \"node-resolver-xjxgr\" (UID: \"01e9bde7-f187-4fe4-b933-e31dfc8de92b\") " pod="openshift-dns/node-resolver-xjxgr" Jul 16 04:39:09.883263 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.882854 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpcdk\" (UniqueName: \"kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk\") pod \"network-check-target-bwl6r\" (UID: \"7c288293-bb92-454b-b0de-91d7317cda4b\") " pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:09.883263 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.882904 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-kubernetes\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.883263 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.882930 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-socket-dir\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.883263 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.882954 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-host-var-lib-kubelet\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.883263 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.882975 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-multus-cni-dir\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.883263 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.882978 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs\") pod \"network-metrics-daemon-7g5n5\" (UID: \"2f51d802-9618-4e88-8fc8-78b2a3788356\") " pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:09.883263 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883034 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-host\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.883263 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883059 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-tuned\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.883263 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883089 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-run-ovn\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.883263 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883143 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-host-var-lib-cni-multus\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.884023 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883156 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-host-var-lib-kubelet\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.884023 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883179 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-socket-dir\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.884023 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883203 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd-iptables-alerter-script\") pod \"iptables-alerter-47pt2\" (UID: \"e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd\") " pod="openshift-network-operator/iptables-alerter-47pt2" Jul 16 04:39:09.884023 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883244 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0d695a61-7e54-42ac-9251-a3d84eeb3baa-host-var-lib-cni-multus\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.884023 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883295 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-lib-modules\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.884023 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883312 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01e9bde7-f187-4fe4-b933-e31dfc8de92b-tmp-dir\") pod \"node-resolver-xjxgr\" (UID: \"01e9bde7-f187-4fe4-b933-e31dfc8de92b\") " pod="openshift-dns/node-resolver-xjxgr" Jul 16 04:39:09.884023 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883326 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99e2e193-30b8-43ed-9e81-023f10d73b10-cnibin\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.884023 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883355 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99e2e193-30b8-43ed-9e81-023f10d73b10-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.884023 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883410 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5da9fe69-0b36-4198-a5ff-329ddaee7389-kubelet-config\") pod \"global-pull-secret-syncer-g6vzk\" (UID: \"5da9fe69-0b36-4198-a5ff-329ddaee7389\") " pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:09.884023 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883444 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-systemd\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.884023 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883472 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bb26\" (UniqueName: \"kubernetes.io/projected/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-kube-api-access-5bb26\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.884023 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883501 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.884023 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883542 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/95a3a732-50cf-47f5-9629-0b3d241d017a-env-overrides\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.884023 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883570 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-registration-dir\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.884699 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883595 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3b736bea-a20b-4220-b592-3819366c1ebd-konnectivity-ca\") pod \"konnectivity-agent-fnkvt\" (UID: \"3b736bea-a20b-4220-b592-3819366c1ebd\") " pod="kube-system/konnectivity-agent-fnkvt" Jul 16 04:39:09.884699 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883619 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-node-log\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.884699 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883644 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf6c2\" (UniqueName: \"kubernetes.io/projected/95a3a732-50cf-47f5-9629-0b3d241d017a-kube-api-access-nf6c2\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.884699 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.883764 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-registration-dir\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.884699 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.884331 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3b736bea-a20b-4220-b592-3819366c1ebd-konnectivity-ca\") pod \"konnectivity-agent-fnkvt\" (UID: \"3b736bea-a20b-4220-b592-3819366c1ebd\") " pod="kube-system/konnectivity-agent-fnkvt" Jul 16 04:39:09.884699 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.884560 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd-iptables-alerter-script\") pod \"iptables-alerter-47pt2\" (UID: \"e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd\") " pod="openshift-network-operator/iptables-alerter-47pt2" Jul 16 04:39:09.885993 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.885974 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3b736bea-a20b-4220-b592-3819366c1ebd-agent-certs\") pod \"konnectivity-agent-fnkvt\" (UID: \"3b736bea-a20b-4220-b592-3819366c1ebd\") " pod="kube-system/konnectivity-agent-fnkvt" Jul 16 04:39:09.890951 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.890924 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhpgj\" (UniqueName: \"kubernetes.io/projected/0d695a61-7e54-42ac-9251-a3d84eeb3baa-kube-api-access-bhpgj\") pod \"multus-hnbvz\" (UID: \"0d695a61-7e54-42ac-9251-a3d84eeb3baa\") " pod="openshift-multus/multus-hnbvz" Jul 16 04:39:09.891053 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.890955 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgbnh\" (UniqueName: \"kubernetes.io/projected/01e9bde7-f187-4fe4-b933-e31dfc8de92b-kube-api-access-xgbnh\") pod \"node-resolver-xjxgr\" (UID: \"01e9bde7-f187-4fe4-b933-e31dfc8de92b\") " pod="openshift-dns/node-resolver-xjxgr" Jul 16 04:39:09.898137 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.898109 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rttq\" (UniqueName: \"kubernetes.io/projected/e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd-kube-api-access-5rttq\") pod \"iptables-alerter-47pt2\" (UID: \"e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd\") " pod="openshift-network-operator/iptables-alerter-47pt2" Jul 16 04:39:09.899546 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.899525 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rt7z\" (UniqueName: \"kubernetes.io/projected/a4363e93-6e92-45a8-b2b7-d75a44e3fe70-kube-api-access-9rt7z\") pod \"aws-ebs-csi-driver-node-vgnpx\" (UID: \"a4363e93-6e92-45a8-b2b7-d75a44e3fe70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.983907 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.983875 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-tmp\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.983907 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.983909 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/95a3a732-50cf-47f5-9629-0b3d241d017a-ovnkube-script-lib\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.984153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.983939 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-run-systemd\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.984153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.983965 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-var-lib-openvswitch\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.984153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.983988 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-run-ovn-kubernetes\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.984153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984016 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/95a3a732-50cf-47f5-9629-0b3d241d017a-ovnkube-config\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.984153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984042 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-sysconfig\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.984153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984064 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-kubelet\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.984153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984086 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-log-socket\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.984153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984093 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-var-lib-openvswitch\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.984153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984115 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trjwk\" (UniqueName: \"kubernetes.io/projected/99e2e193-30b8-43ed-9e81-023f10d73b10-kube-api-access-trjwk\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.984153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984093 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-run-ovn-kubernetes\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.984153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984130 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-run-systemd\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.984153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984155 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-sysctl-conf\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.984832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984183 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-slash\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.984832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984200 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-sysconfig\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.984832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984210 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-run-openvswitch\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.984832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984240 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/99e2e193-30b8-43ed-9e81-023f10d73b10-os-release\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.984832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984259 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-log-socket\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.984832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984263 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-run\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.984832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984296 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99e2e193-30b8-43ed-9e81-023f10d73b10-cni-binary-copy\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.984832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984319 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-sys\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.984832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984325 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-run-openvswitch\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.984832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984350 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-var-lib-kubelet\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.984832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984355 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/99e2e193-30b8-43ed-9e81-023f10d73b10-os-release\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.984832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984379 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-run-netns\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.984832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984385 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-kubelet\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.984832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984404 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-etc-openvswitch\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.984832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984467 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-sys\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.985575 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984481 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-etc-openvswitch\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.985575 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984515 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-var-lib-kubelet\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.985575 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984551 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-sysctl-conf\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.985575 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984595 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-run-netns\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.985575 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984612 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/95a3a732-50cf-47f5-9629-0b3d241d017a-ovnkube-script-lib\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.985575 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984639 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-slash\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.985575 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984671 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/95a3a732-50cf-47f5-9629-0b3d241d017a-ovn-node-metrics-cert\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.985575 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984808 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-run\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.985575 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.984929 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/95a3a732-50cf-47f5-9629-0b3d241d017a-ovnkube-config\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.985575 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985024 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5da9fe69-0b36-4198-a5ff-329ddaee7389-dbus\") pod \"global-pull-secret-syncer-g6vzk\" (UID: \"5da9fe69-0b36-4198-a5ff-329ddaee7389\") " pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:09.985575 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985033 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99e2e193-30b8-43ed-9e81-023f10d73b10-cni-binary-copy\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.985575 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985052 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-modprobe-d\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.985575 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985077 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-systemd-units\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.985575 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985088 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5da9fe69-0b36-4198-a5ff-329ddaee7389-dbus\") pod \"global-pull-secret-syncer-g6vzk\" (UID: \"5da9fe69-0b36-4198-a5ff-329ddaee7389\") " pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:09.985575 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985102 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-cni-netd\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.986337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985131 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/99e2e193-30b8-43ed-9e81-023f10d73b10-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.986337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985136 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-systemd-units\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.986337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985166 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-cni-netd\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.986337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985167 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-sysctl-d\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.986337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985213 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpcdk\" (UniqueName: \"kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk\") pod \"network-check-target-bwl6r\" (UID: \"7c288293-bb92-454b-b0de-91d7317cda4b\") " pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:09.986337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985235 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-modprobe-d\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.986337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985239 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-kubernetes\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.986337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985277 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-sysctl-d\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.986337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985281 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-kubernetes\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.986337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985292 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs\") pod \"network-metrics-daemon-7g5n5\" (UID: \"2f51d802-9618-4e88-8fc8-78b2a3788356\") " pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:09.986337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985321 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-host\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.986337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985346 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-tuned\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.986337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985369 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-run-ovn\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.986337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985394 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-lib-modules\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.986337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985438 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-host\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.986971 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985417 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99e2e193-30b8-43ed-9e81-023f10d73b10-cnibin\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.986971 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985481 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99e2e193-30b8-43ed-9e81-023f10d73b10-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.986971 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985536 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99e2e193-30b8-43ed-9e81-023f10d73b10-cnibin\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.986971 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985599 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/99e2e193-30b8-43ed-9e81-023f10d73b10-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.986971 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985658 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-run-ovn\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.986971 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985659 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99e2e193-30b8-43ed-9e81-023f10d73b10-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.986971 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985672 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-lib-modules\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.986971 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985699 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5da9fe69-0b36-4198-a5ff-329ddaee7389-kubelet-config\") pod \"global-pull-secret-syncer-g6vzk\" (UID: \"5da9fe69-0b36-4198-a5ff-329ddaee7389\") " pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:09.986971 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985731 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-systemd\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.986971 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985744 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5da9fe69-0b36-4198-a5ff-329ddaee7389-kubelet-config\") pod \"global-pull-secret-syncer-g6vzk\" (UID: \"5da9fe69-0b36-4198-a5ff-329ddaee7389\") " pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:09.986971 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985758 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bb26\" (UniqueName: \"kubernetes.io/projected/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-kube-api-access-5bb26\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.986971 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:09.985773 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:09.986971 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985786 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.986971 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985793 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-systemd\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.986971 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985826 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/95a3a732-50cf-47f5-9629-0b3d241d017a-env-overrides\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.987567 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:09.985839 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs podName:2f51d802-9618-4e88-8fc8-78b2a3788356 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:10.485816716 +0000 UTC m=+3.211285220 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs") pod "network-metrics-daemon-7g5n5" (UID: "2f51d802-9618-4e88-8fc8-78b2a3788356") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:09.987567 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985842 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.987567 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985872 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-node-log\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.987567 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985902 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf6c2\" (UniqueName: \"kubernetes.io/projected/95a3a732-50cf-47f5-9629-0b3d241d017a-kube-api-access-nf6c2\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.987567 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985934 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpjdr\" (UniqueName: \"kubernetes.io/projected/852b5867-4340-4c84-9da8-8e4f02270f0d-kube-api-access-rpjdr\") pod \"node-ca-cngtn\" (UID: \"852b5867-4340-4c84-9da8-8e4f02270f0d\") " pod="openshift-image-registry/node-ca-cngtn" Jul 16 04:39:09.987567 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985961 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4c5z\" (UniqueName: \"kubernetes.io/projected/2f51d802-9618-4e88-8fc8-78b2a3788356-kube-api-access-q4c5z\") pod \"network-metrics-daemon-7g5n5\" (UID: \"2f51d802-9618-4e88-8fc8-78b2a3788356\") " pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:09.987567 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.985989 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret\") pod \"global-pull-secret-syncer-g6vzk\" (UID: \"5da9fe69-0b36-4198-a5ff-329ddaee7389\") " pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:09.987567 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.986014 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-cni-bin\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.987567 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.986040 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/852b5867-4340-4c84-9da8-8e4f02270f0d-host\") pod \"node-ca-cngtn\" (UID: \"852b5867-4340-4c84-9da8-8e4f02270f0d\") " pod="openshift-image-registry/node-ca-cngtn" Jul 16 04:39:09.987567 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.986066 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/852b5867-4340-4c84-9da8-8e4f02270f0d-serviceca\") pod \"node-ca-cngtn\" (UID: \"852b5867-4340-4c84-9da8-8e4f02270f0d\") " pod="openshift-image-registry/node-ca-cngtn" Jul 16 04:39:09.987567 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.986095 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99e2e193-30b8-43ed-9e81-023f10d73b10-system-cni-dir\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.987567 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:09.986114 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:09.987567 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.986124 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/99e2e193-30b8-43ed-9e81-023f10d73b10-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.988123 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:09.986179 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret podName:5da9fe69-0b36-4198-a5ff-329ddaee7389 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:10.486161738 +0000 UTC m=+3.211630478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret") pod "global-pull-secret-syncer-g6vzk" (UID: "5da9fe69-0b36-4198-a5ff-329ddaee7389") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:09.988123 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.986206 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-node-log\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.988123 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.986231 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/95a3a732-50cf-47f5-9629-0b3d241d017a-env-overrides\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.988123 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.986261 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95a3a732-50cf-47f5-9629-0b3d241d017a-host-cni-bin\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.988123 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.986287 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/852b5867-4340-4c84-9da8-8e4f02270f0d-host\") pod \"node-ca-cngtn\" (UID: \"852b5867-4340-4c84-9da8-8e4f02270f0d\") " pod="openshift-image-registry/node-ca-cngtn" Jul 16 04:39:09.988123 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.986308 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99e2e193-30b8-43ed-9e81-023f10d73b10-system-cni-dir\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.988123 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.986675 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/99e2e193-30b8-43ed-9e81-023f10d73b10-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.988123 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.986733 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/852b5867-4340-4c84-9da8-8e4f02270f0d-serviceca\") pod \"node-ca-cngtn\" (UID: \"852b5867-4340-4c84-9da8-8e4f02270f0d\") " pod="openshift-image-registry/node-ca-cngtn" Jul 16 04:39:09.988123 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.986875 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-tmp\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.988545 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.988160 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/95a3a732-50cf-47f5-9629-0b3d241d017a-ovn-node-metrics-cert\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:09.988545 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.988391 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-etc-tuned\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.993011 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:09.992986 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:39:09.993127 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:09.993018 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:39:09.993127 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:09.993033 2581 projected.go:196] Error preparing data for projected volume kube-api-access-xpcdk for pod openshift-network-diagnostics/network-check-target-bwl6r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:09.993127 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:09.993083 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk podName:7c288293-bb92-454b-b0de-91d7317cda4b nodeName:}" failed. No retries permitted until 2026-07-16 04:39:10.493068299 +0000 UTC m=+3.218536802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xpcdk" (UniqueName: "kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk") pod "network-check-target-bwl6r" (UID: "7c288293-bb92-454b-b0de-91d7317cda4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:09.994064 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.994037 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trjwk\" (UniqueName: \"kubernetes.io/projected/99e2e193-30b8-43ed-9e81-023f10d73b10-kube-api-access-trjwk\") pod \"multus-additional-cni-plugins-m9m2p\" (UID: \"99e2e193-30b8-43ed-9e81-023f10d73b10\") " pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:09.994874 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.994846 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4c5z\" (UniqueName: \"kubernetes.io/projected/2f51d802-9618-4e88-8fc8-78b2a3788356-kube-api-access-q4c5z\") pod \"network-metrics-daemon-7g5n5\" (UID: \"2f51d802-9618-4e88-8fc8-78b2a3788356\") " pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:09.995517 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.995471 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpjdr\" (UniqueName: \"kubernetes.io/projected/852b5867-4340-4c84-9da8-8e4f02270f0d-kube-api-access-rpjdr\") pod \"node-ca-cngtn\" (UID: \"852b5867-4340-4c84-9da8-8e4f02270f0d\") " pod="openshift-image-registry/node-ca-cngtn" Jul 16 04:39:09.995777 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.995750 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bb26\" (UniqueName: \"kubernetes.io/projected/515d6242-fd45-4b4e-b6e9-a1918a6d14f0-kube-api-access-5bb26\") pod \"tuned-lplz2\" (UID: \"515d6242-fd45-4b4e-b6e9-a1918a6d14f0\") " pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:09.998735 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.998677 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" Jul 16 04:39:09.999711 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:09.999686 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf6c2\" (UniqueName: \"kubernetes.io/projected/95a3a732-50cf-47f5-9629-0b3d241d017a-kube-api-access-nf6c2\") pod \"ovnkube-node-lrbbv\" (UID: \"95a3a732-50cf-47f5-9629-0b3d241d017a\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:10.005667 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.005647 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xjxgr" Jul 16 04:39:10.010988 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:10.010478 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4363e93_6e92_45a8_b2b7_d75a44e3fe70.slice/crio-5adf5aa5f05ef067097cfeef09840c63ef37b2199f813928bd1124ed010d1dae WatchSource:0}: Error finding container 5adf5aa5f05ef067097cfeef09840c63ef37b2199f813928bd1124ed010d1dae: Status 404 returned error can't find the container with id 5adf5aa5f05ef067097cfeef09840c63ef37b2199f813928bd1124ed010d1dae Jul 16 04:39:10.013043 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.013018 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hnbvz" Jul 16 04:39:10.017438 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:10.017415 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e9bde7_f187_4fe4_b933_e31dfc8de92b.slice/crio-8092c0a3d3c1080c0ec10e46313cc863b1c81f35e3346f425fde1387999f02df WatchSource:0}: Error finding container 8092c0a3d3c1080c0ec10e46313cc863b1c81f35e3346f425fde1387999f02df: Status 404 returned error can't find the container with id 8092c0a3d3c1080c0ec10e46313cc863b1c81f35e3346f425fde1387999f02df Jul 16 04:39:10.018820 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.018800 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-47pt2" Jul 16 04:39:10.022806 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:10.022782 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d695a61_7e54_42ac_9251_a3d84eeb3baa.slice/crio-3aa2f1b2d48fffb3631f3ffeb7686277be3e285eaedd0d86923248b00fc4efde WatchSource:0}: Error finding container 3aa2f1b2d48fffb3631f3ffeb7686277be3e285eaedd0d86923248b00fc4efde: Status 404 returned error can't find the container with id 3aa2f1b2d48fffb3631f3ffeb7686277be3e285eaedd0d86923248b00fc4efde Jul 16 04:39:10.026768 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.026670 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fnkvt" Jul 16 04:39:10.028750 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:10.028724 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2adfeb9_5c71_41bb_8716_ef1f3f2e2cbd.slice/crio-190e2a15f37d046884a128eab95bfb975b1f7d6c759af0ac58e10e16a013e7fa WatchSource:0}: Error finding container 190e2a15f37d046884a128eab95bfb975b1f7d6c759af0ac58e10e16a013e7fa: Status 404 returned error can't find the container with id 190e2a15f37d046884a128eab95bfb975b1f7d6c759af0ac58e10e16a013e7fa Jul 16 04:39:10.032329 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.032308 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cngtn" Jul 16 04:39:10.037465 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:10.037444 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b736bea_a20b_4220_b592_3819366c1ebd.slice/crio-86bbb5ba73ccc074edd5408e7dd971b6c27e3460759bd9ab0ff0a7fb08b9be21 WatchSource:0}: Error finding container 86bbb5ba73ccc074edd5408e7dd971b6c27e3460759bd9ab0ff0a7fb08b9be21: Status 404 returned error can't find the container with id 86bbb5ba73ccc074edd5408e7dd971b6c27e3460759bd9ab0ff0a7fb08b9be21 Jul 16 04:39:10.037619 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.037599 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lplz2" Jul 16 04:39:10.043782 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.043764 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m9m2p" Jul 16 04:39:10.044288 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:10.044264 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod852b5867_4340_4c84_9da8_8e4f02270f0d.slice/crio-3d00510ecd1c72cdb17eb320fb843568db9618b76c6472033cc1aaff382d2f67 WatchSource:0}: Error finding container 3d00510ecd1c72cdb17eb320fb843568db9618b76c6472033cc1aaff382d2f67: Status 404 returned error can't find the container with id 3d00510ecd1c72cdb17eb320fb843568db9618b76c6472033cc1aaff382d2f67 Jul 16 04:39:10.048215 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:10.048192 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod515d6242_fd45_4b4e_b6e9_a1918a6d14f0.slice/crio-396b6bd72a20f23f12e2ccdd085a94711ea70ada1baf5ef29cca9558ceb0dbb6 WatchSource:0}: Error finding container 396b6bd72a20f23f12e2ccdd085a94711ea70ada1baf5ef29cca9558ceb0dbb6: Status 404 returned error can't find the container with id 396b6bd72a20f23f12e2ccdd085a94711ea70ada1baf5ef29cca9558ceb0dbb6 Jul 16 04:39:10.048309 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.048295 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:10.055755 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:10.055732 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99e2e193_30b8_43ed_9e81_023f10d73b10.slice/crio-df5dcaf89a1d427abe58f1871d31c0a8336978fc405283e9a90bbf1b247e0bb0 WatchSource:0}: Error finding container df5dcaf89a1d427abe58f1871d31c0a8336978fc405283e9a90bbf1b247e0bb0: Status 404 returned error can't find the container with id df5dcaf89a1d427abe58f1871d31c0a8336978fc405283e9a90bbf1b247e0bb0 Jul 16 04:39:10.061998 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:10.061975 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95a3a732_50cf_47f5_9629_0b3d241d017a.slice/crio-a4802b98cfc4d062fbfd7ecc0d29a58205e18836cf7202cdc3ad10d710f1d2f7 WatchSource:0}: Error finding container a4802b98cfc4d062fbfd7ecc0d29a58205e18836cf7202cdc3ad10d710f1d2f7: Status 404 returned error can't find the container with id a4802b98cfc4d062fbfd7ecc0d29a58205e18836cf7202cdc3ad10d710f1d2f7 Jul 16 04:39:10.170836 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.170770 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9m2p" event={"ID":"99e2e193-30b8-43ed-9e81-023f10d73b10","Type":"ContainerStarted","Data":"df5dcaf89a1d427abe58f1871d31c0a8336978fc405283e9a90bbf1b247e0bb0"} Jul 16 04:39:10.172802 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.172718 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lplz2" event={"ID":"515d6242-fd45-4b4e-b6e9-a1918a6d14f0","Type":"ContainerStarted","Data":"396b6bd72a20f23f12e2ccdd085a94711ea70ada1baf5ef29cca9558ceb0dbb6"} Jul 16 04:39:10.174337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.174263 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cngtn" event={"ID":"852b5867-4340-4c84-9da8-8e4f02270f0d","Type":"ContainerStarted","Data":"3d00510ecd1c72cdb17eb320fb843568db9618b76c6472033cc1aaff382d2f67"} Jul 16 04:39:10.175921 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.175880 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fnkvt" event={"ID":"3b736bea-a20b-4220-b592-3819366c1ebd","Type":"ContainerStarted","Data":"86bbb5ba73ccc074edd5408e7dd971b6c27e3460759bd9ab0ff0a7fb08b9be21"} Jul 16 04:39:10.177801 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.177775 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-47pt2" event={"ID":"e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd","Type":"ContainerStarted","Data":"190e2a15f37d046884a128eab95bfb975b1f7d6c759af0ac58e10e16a013e7fa"} Jul 16 04:39:10.179210 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.179185 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal" event={"ID":"0991fafe223622889a10e1583a4386ad","Type":"ContainerStarted","Data":"ad9fac6db4a0a6f08bd330f1a0df3426da1d68fe1adde20f0dd37a9417742190"} Jul 16 04:39:10.180828 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.180803 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal" event={"ID":"40de15c40dab007c9ee75dd5f90e7e7c","Type":"ContainerStarted","Data":"14e0694b5e816f9cc02a4f155c6b71c3ad23d931e5d25629d7dfd8bab5661205"} Jul 16 04:39:10.185215 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.185163 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" event={"ID":"95a3a732-50cf-47f5-9629-0b3d241d017a","Type":"ContainerStarted","Data":"a4802b98cfc4d062fbfd7ecc0d29a58205e18836cf7202cdc3ad10d710f1d2f7"} Jul 16 04:39:10.187602 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.187544 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hnbvz" event={"ID":"0d695a61-7e54-42ac-9251-a3d84eeb3baa","Type":"ContainerStarted","Data":"3aa2f1b2d48fffb3631f3ffeb7686277be3e285eaedd0d86923248b00fc4efde"} Jul 16 04:39:10.189257 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.189202 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xjxgr" event={"ID":"01e9bde7-f187-4fe4-b933-e31dfc8de92b","Type":"ContainerStarted","Data":"8092c0a3d3c1080c0ec10e46313cc863b1c81f35e3346f425fde1387999f02df"} Jul 16 04:39:10.190925 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.190811 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" event={"ID":"a4363e93-6e92-45a8-b2b7-d75a44e3fe70","Type":"ContainerStarted","Data":"5adf5aa5f05ef067097cfeef09840c63ef37b2199f813928bd1124ed010d1dae"} Jul 16 04:39:10.490296 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.490197 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs\") pod \"network-metrics-daemon-7g5n5\" (UID: \"2f51d802-9618-4e88-8fc8-78b2a3788356\") " pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:10.490296 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.490260 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret\") pod \"global-pull-secret-syncer-g6vzk\" (UID: \"5da9fe69-0b36-4198-a5ff-329ddaee7389\") " pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:10.490540 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:10.490409 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:10.490540 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:10.490481 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret podName:5da9fe69-0b36-4198-a5ff-329ddaee7389 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:11.49046048 +0000 UTC m=+4.215928988 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret") pod "global-pull-secret-syncer-g6vzk" (UID: "5da9fe69-0b36-4198-a5ff-329ddaee7389") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:10.490965 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:10.490818 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:10.490965 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:10.490889 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs podName:2f51d802-9618-4e88-8fc8-78b2a3788356 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:11.490870529 +0000 UTC m=+4.216339055 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs") pod "network-metrics-daemon-7g5n5" (UID: "2f51d802-9618-4e88-8fc8-78b2a3788356") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:10.591529 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.591488 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpcdk\" (UniqueName: \"kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk\") pod \"network-check-target-bwl6r\" (UID: \"7c288293-bb92-454b-b0de-91d7317cda4b\") " pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:10.591705 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:10.591678 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:39:10.591801 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:10.591716 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:39:10.591801 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:10.591731 2581 projected.go:196] Error preparing data for projected volume kube-api-access-xpcdk for pod openshift-network-diagnostics/network-check-target-bwl6r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:10.591905 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:10.591806 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk podName:7c288293-bb92-454b-b0de-91d7317cda4b nodeName:}" failed. No retries permitted until 2026-07-16 04:39:11.591792441 +0000 UTC m=+4.317260957 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xpcdk" (UniqueName: "kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk") pod "network-check-target-bwl6r" (UID: "7c288293-bb92-454b-b0de-91d7317cda4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:10.817252 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.817162 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 04:34:09 +0000 UTC" deadline="2028-02-14 03:11:12.62458755 +0000 UTC" Jul 16 04:39:10.817252 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.817197 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13870h32m1.807395602s" Jul 16 04:39:10.833560 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:10.833527 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:10.833694 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:10.833676 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7g5n5" podUID="2f51d802-9618-4e88-8fc8-78b2a3788356" Jul 16 04:39:11.498452 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:11.498411 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret\") pod \"global-pull-secret-syncer-g6vzk\" (UID: \"5da9fe69-0b36-4198-a5ff-329ddaee7389\") " pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:11.498968 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:11.498519 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs\") pod \"network-metrics-daemon-7g5n5\" (UID: \"2f51d802-9618-4e88-8fc8-78b2a3788356\") " pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:11.498968 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:11.498670 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:11.498968 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:11.498736 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs podName:2f51d802-9618-4e88-8fc8-78b2a3788356 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:13.498716163 +0000 UTC m=+6.224184667 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs") pod "network-metrics-daemon-7g5n5" (UID: "2f51d802-9618-4e88-8fc8-78b2a3788356") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:11.499199 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:11.499177 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:11.499276 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:11.499236 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret podName:5da9fe69-0b36-4198-a5ff-329ddaee7389 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:13.499220415 +0000 UTC m=+6.224688918 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret") pod "global-pull-secret-syncer-g6vzk" (UID: "5da9fe69-0b36-4198-a5ff-329ddaee7389") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:11.600250 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:11.599494 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpcdk\" (UniqueName: \"kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk\") pod \"network-check-target-bwl6r\" (UID: \"7c288293-bb92-454b-b0de-91d7317cda4b\") " pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:11.600250 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:11.599696 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:39:11.600250 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:11.599717 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:39:11.600250 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:11.599731 2581 projected.go:196] Error preparing data for projected volume kube-api-access-xpcdk for pod openshift-network-diagnostics/network-check-target-bwl6r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:11.600250 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:11.599787 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk podName:7c288293-bb92-454b-b0de-91d7317cda4b nodeName:}" failed. No retries permitted until 2026-07-16 04:39:13.59976984 +0000 UTC m=+6.325238348 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xpcdk" (UniqueName: "kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk") pod "network-check-target-bwl6r" (UID: "7c288293-bb92-454b-b0de-91d7317cda4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:11.818168 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:11.818043 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 04:34:09 +0000 UTC" deadline="2027-12-29 09:44:49.753282364 +0000 UTC" Jul 16 04:39:11.818168 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:11.818109 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12749h5m37.935177175s" Jul 16 04:39:11.835361 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:11.835336 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:11.835517 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:11.835463 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwl6r" podUID="7c288293-bb92-454b-b0de-91d7317cda4b" Jul 16 04:39:11.835922 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:11.835901 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:11.836025 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:11.836003 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6vzk" podUID="5da9fe69-0b36-4198-a5ff-329ddaee7389" Jul 16 04:39:12.833092 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:12.832897 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:12.833092 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:12.833033 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7g5n5" podUID="2f51d802-9618-4e88-8fc8-78b2a3788356" Jul 16 04:39:13.514616 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:13.514584 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs\") pod \"network-metrics-daemon-7g5n5\" (UID: \"2f51d802-9618-4e88-8fc8-78b2a3788356\") " pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:13.514793 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:13.514634 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret\") pod \"global-pull-secret-syncer-g6vzk\" (UID: \"5da9fe69-0b36-4198-a5ff-329ddaee7389\") " pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:13.514793 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:13.514759 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:13.514912 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:13.514816 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret podName:5da9fe69-0b36-4198-a5ff-329ddaee7389 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:17.514798214 +0000 UTC m=+10.240266719 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret") pod "global-pull-secret-syncer-g6vzk" (UID: "5da9fe69-0b36-4198-a5ff-329ddaee7389") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:13.514912 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:13.514880 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:13.515019 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:13.514914 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs podName:2f51d802-9618-4e88-8fc8-78b2a3788356 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:17.514902221 +0000 UTC m=+10.240370736 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs") pod "network-metrics-daemon-7g5n5" (UID: "2f51d802-9618-4e88-8fc8-78b2a3788356") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:13.616010 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:13.615973 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpcdk\" (UniqueName: \"kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk\") pod \"network-check-target-bwl6r\" (UID: \"7c288293-bb92-454b-b0de-91d7317cda4b\") " pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:13.616195 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:13.616177 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:39:13.616260 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:13.616204 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:39:13.616260 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:13.616217 2581 projected.go:196] Error preparing data for projected volume kube-api-access-xpcdk for pod openshift-network-diagnostics/network-check-target-bwl6r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:13.616351 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:13.616280 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk podName:7c288293-bb92-454b-b0de-91d7317cda4b nodeName:}" failed. No retries permitted until 2026-07-16 04:39:17.616259995 +0000 UTC m=+10.341728512 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xpcdk" (UniqueName: "kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk") pod "network-check-target-bwl6r" (UID: "7c288293-bb92-454b-b0de-91d7317cda4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:13.834407 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:13.833705 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:13.834407 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:13.833844 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwl6r" podUID="7c288293-bb92-454b-b0de-91d7317cda4b" Jul 16 04:39:13.834407 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:13.834223 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:13.834407 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:13.834315 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6vzk" podUID="5da9fe69-0b36-4198-a5ff-329ddaee7389" Jul 16 04:39:14.833156 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:14.832776 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:14.833156 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:14.832872 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7g5n5" podUID="2f51d802-9618-4e88-8fc8-78b2a3788356" Jul 16 04:39:15.833009 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:15.832917 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:15.833473 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:15.832922 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:15.833473 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:15.833039 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwl6r" podUID="7c288293-bb92-454b-b0de-91d7317cda4b" Jul 16 04:39:15.833473 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:15.833150 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6vzk" podUID="5da9fe69-0b36-4198-a5ff-329ddaee7389" Jul 16 04:39:16.833684 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:16.833590 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:16.834056 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:16.833744 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7g5n5" podUID="2f51d802-9618-4e88-8fc8-78b2a3788356" Jul 16 04:39:17.549678 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:17.549645 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs\") pod \"network-metrics-daemon-7g5n5\" (UID: \"2f51d802-9618-4e88-8fc8-78b2a3788356\") " pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:17.549863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:17.549688 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret\") pod \"global-pull-secret-syncer-g6vzk\" (UID: \"5da9fe69-0b36-4198-a5ff-329ddaee7389\") " pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:17.549863 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:17.549780 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:17.549863 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:17.549810 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:17.549863 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:17.549838 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret podName:5da9fe69-0b36-4198-a5ff-329ddaee7389 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:25.549819961 +0000 UTC m=+18.275288483 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret") pod "global-pull-secret-syncer-g6vzk" (UID: "5da9fe69-0b36-4198-a5ff-329ddaee7389") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:17.550083 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:17.549873 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs podName:2f51d802-9618-4e88-8fc8-78b2a3788356 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:25.549854939 +0000 UTC m=+18.275323455 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs") pod "network-metrics-daemon-7g5n5" (UID: "2f51d802-9618-4e88-8fc8-78b2a3788356") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:17.650283 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:17.650230 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpcdk\" (UniqueName: \"kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk\") pod \"network-check-target-bwl6r\" (UID: \"7c288293-bb92-454b-b0de-91d7317cda4b\") " pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:17.650468 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:17.650434 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:39:17.650468 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:17.650451 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:39:17.650468 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:17.650462 2581 projected.go:196] Error preparing data for projected volume kube-api-access-xpcdk for pod openshift-network-diagnostics/network-check-target-bwl6r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:17.650659 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:17.650540 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk podName:7c288293-bb92-454b-b0de-91d7317cda4b nodeName:}" failed. No retries permitted until 2026-07-16 04:39:25.650520465 +0000 UTC m=+18.375988967 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xpcdk" (UniqueName: "kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk") pod "network-check-target-bwl6r" (UID: "7c288293-bb92-454b-b0de-91d7317cda4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:17.837087 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:17.837020 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:17.837458 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:17.837134 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwl6r" podUID="7c288293-bb92-454b-b0de-91d7317cda4b" Jul 16 04:39:17.837458 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:17.837243 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:17.837458 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:17.837329 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6vzk" podUID="5da9fe69-0b36-4198-a5ff-329ddaee7389" Jul 16 04:39:18.833529 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:18.833350 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:18.833830 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:18.833796 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7g5n5" podUID="2f51d802-9618-4e88-8fc8-78b2a3788356" Jul 16 04:39:19.833089 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:19.832973 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:19.833089 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:19.832977 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:19.833643 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:19.833108 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwl6r" podUID="7c288293-bb92-454b-b0de-91d7317cda4b" Jul 16 04:39:19.833643 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:19.833164 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6vzk" podUID="5da9fe69-0b36-4198-a5ff-329ddaee7389" Jul 16 04:39:20.833605 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:20.833571 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:20.833959 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:20.833697 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7g5n5" podUID="2f51d802-9618-4e88-8fc8-78b2a3788356" Jul 16 04:39:21.832794 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:21.832761 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:21.832959 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:21.832893 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwl6r" podUID="7c288293-bb92-454b-b0de-91d7317cda4b" Jul 16 04:39:21.833028 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:21.832952 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:21.833097 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:21.833076 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6vzk" podUID="5da9fe69-0b36-4198-a5ff-329ddaee7389" Jul 16 04:39:22.833225 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:22.833194 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:22.833657 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:22.833321 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7g5n5" podUID="2f51d802-9618-4e88-8fc8-78b2a3788356" Jul 16 04:39:23.832893 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:23.832854 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:23.833092 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:23.832868 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:23.833092 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:23.832992 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwl6r" podUID="7c288293-bb92-454b-b0de-91d7317cda4b" Jul 16 04:39:23.833092 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:23.833054 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6vzk" podUID="5da9fe69-0b36-4198-a5ff-329ddaee7389" Jul 16 04:39:24.833270 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:24.833187 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:24.833765 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:24.833324 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7g5n5" podUID="2f51d802-9618-4e88-8fc8-78b2a3788356" Jul 16 04:39:25.609552 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:25.609496 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs\") pod \"network-metrics-daemon-7g5n5\" (UID: \"2f51d802-9618-4e88-8fc8-78b2a3788356\") " pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:25.609742 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:25.609568 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret\") pod \"global-pull-secret-syncer-g6vzk\" (UID: \"5da9fe69-0b36-4198-a5ff-329ddaee7389\") " pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:25.609742 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:25.609668 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:25.609851 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:25.609746 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs podName:2f51d802-9618-4e88-8fc8-78b2a3788356 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:41.60972762 +0000 UTC m=+34.335196136 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs") pod "network-metrics-daemon-7g5n5" (UID: "2f51d802-9618-4e88-8fc8-78b2a3788356") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:25.609851 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:25.609672 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:25.609851 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:25.609828 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret podName:5da9fe69-0b36-4198-a5ff-329ddaee7389 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:41.609810097 +0000 UTC m=+34.335278597 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret") pod "global-pull-secret-syncer-g6vzk" (UID: "5da9fe69-0b36-4198-a5ff-329ddaee7389") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:25.710406 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:25.710365 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpcdk\" (UniqueName: \"kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk\") pod \"network-check-target-bwl6r\" (UID: \"7c288293-bb92-454b-b0de-91d7317cda4b\") " pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:25.710602 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:25.710582 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:39:25.710652 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:25.710608 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:39:25.710652 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:25.710621 2581 projected.go:196] Error preparing data for projected volume kube-api-access-xpcdk for pod openshift-network-diagnostics/network-check-target-bwl6r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:25.710721 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:25.710685 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk podName:7c288293-bb92-454b-b0de-91d7317cda4b nodeName:}" failed. No retries permitted until 2026-07-16 04:39:41.710666711 +0000 UTC m=+34.436135233 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xpcdk" (UniqueName: "kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk") pod "network-check-target-bwl6r" (UID: "7c288293-bb92-454b-b0de-91d7317cda4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:25.833638 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:25.833607 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:25.834038 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:25.833732 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwl6r" podUID="7c288293-bb92-454b-b0de-91d7317cda4b" Jul 16 04:39:25.834038 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:25.833772 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:25.834038 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:25.833879 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6vzk" podUID="5da9fe69-0b36-4198-a5ff-329ddaee7389" Jul 16 04:39:26.833387 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:26.833354 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:26.833596 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:26.833485 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7g5n5" podUID="2f51d802-9618-4e88-8fc8-78b2a3788356" Jul 16 04:39:27.834426 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:27.834394 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:27.834884 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:27.834495 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwl6r" podUID="7c288293-bb92-454b-b0de-91d7317cda4b" Jul 16 04:39:27.834884 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:27.834594 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:27.834884 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:27.834703 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6vzk" podUID="5da9fe69-0b36-4198-a5ff-329ddaee7389" Jul 16 04:39:28.833013 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:28.832986 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:28.833124 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:28.833106 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7g5n5" podUID="2f51d802-9618-4e88-8fc8-78b2a3788356" Jul 16 04:39:29.244963 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:29.244929 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lplz2" event={"ID":"515d6242-fd45-4b4e-b6e9-a1918a6d14f0","Type":"ContainerStarted","Data":"2c41b39ab6996183fbbeda53251056091aeaf9fdab27658587100864e0f50dfa"} Jul 16 04:39:29.247678 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:29.247634 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal" event={"ID":"40de15c40dab007c9ee75dd5f90e7e7c","Type":"ContainerStarted","Data":"93f2d70b7961fc53605f6c438413ab5089a7f9f09d93f6de947587d1759dc05e"} Jul 16 04:39:29.250894 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:29.250847 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" event={"ID":"95a3a732-50cf-47f5-9629-0b3d241d017a","Type":"ContainerStarted","Data":"f79d4459eea07544866fe44c46bd23705156ba85915f87b8cef9ae864790ab48"} Jul 16 04:39:29.273646 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:29.273587 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lplz2" podStartSLOduration=2.521055769 podStartE2EDuration="21.273567699s" totalImagesPullingTime="18.75251193s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:29.273103741 +0000 UTC m=+21.998572264" watchObservedRunningTime="2026-07-16 04:39:29.273567699 +0000 UTC m=+21.999036219" Jul 16 04:39:29.832970 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:29.832880 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:29.833120 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:29.832989 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwl6r" podUID="7c288293-bb92-454b-b0de-91d7317cda4b" Jul 16 04:39:29.833120 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:29.833045 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:29.833233 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:29.833135 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6vzk" podUID="5da9fe69-0b36-4198-a5ff-329ddaee7389" Jul 16 04:39:30.255045 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.255006 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xjxgr" event={"ID":"01e9bde7-f187-4fe4-b933-e31dfc8de92b","Type":"ContainerStarted","Data":"d565a5e89de164fddb0774695479097d1b16e5a975a57b04294afc2ba461319f"} Jul 16 04:39:30.256677 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.256652 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" event={"ID":"a4363e93-6e92-45a8-b2b7-d75a44e3fe70","Type":"ContainerStarted","Data":"46e837cabc6a7d6575f3747a4d6a5b666ce5465e0968895ef8bd2e8fbb0a208a"} Jul 16 04:39:30.258348 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.258316 2581 generic.go:356] "Generic (PLEG): container finished" podID="99e2e193-30b8-43ed-9e81-023f10d73b10" containerID="e81f2867730b96dfd52e17fe4856db15f3f72cea84d47636f2713dae893d3c4b" exitCode=0 Jul 16 04:39:30.258435 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.258360 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9m2p" event={"ID":"99e2e193-30b8-43ed-9e81-023f10d73b10","Type":"ContainerDied","Data":"e81f2867730b96dfd52e17fe4856db15f3f72cea84d47636f2713dae893d3c4b"} Jul 16 04:39:30.260028 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.259954 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cngtn" event={"ID":"852b5867-4340-4c84-9da8-8e4f02270f0d","Type":"ContainerStarted","Data":"48c0e6fe8b73b997e1f40899aa4e0e1b3f7b2521e256b25cb1ec45336a6c5214"} Jul 16 04:39:30.261622 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.261598 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fnkvt" event={"ID":"3b736bea-a20b-4220-b592-3819366c1ebd","Type":"ContainerStarted","Data":"f0408b612ec6cd89b2557d98477047c140dff66b4bce2cb7c7cb86f45b78508b"} Jul 16 04:39:30.264755 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.264729 2581 generic.go:356] "Generic (PLEG): container finished" podID="0991fafe223622889a10e1583a4386ad" containerID="2c7fad298c1dad7a4da1659b4d4a1927c84489fe02c872924f7a457beb6c0254" exitCode=0 Jul 16 04:39:30.264839 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.264807 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal" event={"ID":"0991fafe223622889a10e1583a4386ad","Type":"ContainerDied","Data":"2c7fad298c1dad7a4da1659b4d4a1927c84489fe02c872924f7a457beb6c0254"} Jul 16 04:39:30.267996 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.267977 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrbbv_95a3a732-50cf-47f5-9629-0b3d241d017a/ovn-acl-logging/0.log" Jul 16 04:39:30.268326 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.268307 2581 generic.go:356] "Generic (PLEG): container finished" podID="95a3a732-50cf-47f5-9629-0b3d241d017a" containerID="d018f4f2fcdd12a2ded930669eec40143e3a3c6466befbcc95dd157c0ef135b4" exitCode=1 Jul 16 04:39:30.268424 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.268371 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" event={"ID":"95a3a732-50cf-47f5-9629-0b3d241d017a","Type":"ContainerStarted","Data":"33788da5ac85658e49c6cb175da2934816f6aabcca53a42d2d44fb844b719877"} Jul 16 04:39:30.268488 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.268422 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" event={"ID":"95a3a732-50cf-47f5-9629-0b3d241d017a","Type":"ContainerStarted","Data":"51febb2b562a11e534d3582dd735a6c1ef41cfe1f678867de8e385222940a4b8"} Jul 16 04:39:30.268488 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.268468 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" event={"ID":"95a3a732-50cf-47f5-9629-0b3d241d017a","Type":"ContainerStarted","Data":"9b4a8cbfedea72aaf752bad95d6207918124986b6ab4a5a7dc5d483bd5cd0f6f"} Jul 16 04:39:30.268607 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.268524 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" event={"ID":"95a3a732-50cf-47f5-9629-0b3d241d017a","Type":"ContainerStarted","Data":"29016339cf086d1e3c0ac2a9fcb898d82831d2126e43dc7f9b16d9548f7672ec"} Jul 16 04:39:30.268607 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.268559 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" event={"ID":"95a3a732-50cf-47f5-9629-0b3d241d017a","Type":"ContainerDied","Data":"d018f4f2fcdd12a2ded930669eec40143e3a3c6466befbcc95dd157c0ef135b4"} Jul 16 04:39:30.269900 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.269847 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hnbvz" event={"ID":"0d695a61-7e54-42ac-9251-a3d84eeb3baa","Type":"ContainerStarted","Data":"1513ab5ebb692ab3b6f18a8b83ed612b60f0ea0db54e9d891cfa46772df292db"} Jul 16 04:39:30.279706 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.279670 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal" podStartSLOduration=21.279658317 podStartE2EDuration="21.279658317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:39:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:29.298048299 +0000 UTC m=+22.023516818" watchObservedRunningTime="2026-07-16 04:39:30.279658317 +0000 UTC m=+23.005126838" Jul 16 04:39:30.280116 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.280077 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xjxgr" podStartSLOduration=7.721981094 podStartE2EDuration="22.280062168s" totalImagesPullingTime="14.558081074s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:30.279888977 +0000 UTC m=+23.005357513" watchObservedRunningTime="2026-07-16 04:39:30.280062168 +0000 UTC m=+23.005530689" Jul 16 04:39:30.350290 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.350243 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hnbvz" podStartSLOduration=3.106422896 podStartE2EDuration="22.350228839s" totalImagesPullingTime="19.243805943s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:30.31487376 +0000 UTC m=+23.040342293" watchObservedRunningTime="2026-07-16 04:39:30.350228839 +0000 UTC m=+23.075697359" Jul 16 04:39:30.350457 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.350369 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fnkvt" podStartSLOduration=3.503190355 podStartE2EDuration="22.350366209s" totalImagesPullingTime="18.847175854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:30.349414315 +0000 UTC m=+23.074882845" watchObservedRunningTime="2026-07-16 04:39:30.350366209 +0000 UTC m=+23.075834729" Jul 16 04:39:30.373945 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.373901 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cngtn" podStartSLOduration=13.255447221 podStartE2EDuration="22.373886791s" totalImagesPullingTime="9.11843957s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:30.373777796 +0000 UTC m=+23.099246318" watchObservedRunningTime="2026-07-16 04:39:30.373886791 +0000 UTC m=+23.099355312" Jul 16 04:39:30.731056 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.731026 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 04:39:30.833485 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.833446 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:30.833695 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:30.833593 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7g5n5" podUID="2f51d802-9618-4e88-8fc8-78b2a3788356" Jul 16 04:39:30.977705 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.977617 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fnkvt" Jul 16 04:39:30.978293 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:30.978276 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fnkvt" Jul 16 04:39:31.176779 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:31.176670 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T04:39:30.731050278Z","UUID":"602c28f3-b7bb-4fcd-9092-908d96e48138","Handler":null,"Name":"","Endpoint":""} Jul 16 04:39:31.178365 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:31.178347 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 04:39:31.178481 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:31.178383 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 04:39:31.274205 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:31.273834 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-47pt2" event={"ID":"e2adfeb9-5c71-41bb-8716-ef1f3f2e2cbd","Type":"ContainerStarted","Data":"9963a2b19314a5c2e87f942849a3520234567eb5f4b244c03a14a17560eb2c2b"} Jul 16 04:39:31.276032 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:31.276002 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal" event={"ID":"0991fafe223622889a10e1583a4386ad","Type":"ContainerStarted","Data":"2a516df8c5676db14adb42a9ebfae34127a6adfefd4d5b816bccd17b913ff440"} Jul 16 04:39:31.278837 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:31.278804 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" event={"ID":"a4363e93-6e92-45a8-b2b7-d75a44e3fe70","Type":"ContainerStarted","Data":"02f4ff36f240773648b3d2c240e87d9e4f891d3ee570135d818ed8a5a9c044cf"} Jul 16 04:39:31.279145 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:31.279001 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fnkvt" Jul 16 04:39:31.279595 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:31.279577 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fnkvt" Jul 16 04:39:31.341179 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:31.341111 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-47pt2" podStartSLOduration=8.794306439 podStartE2EDuration="23.341088502s" totalImagesPullingTime="14.546782063s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:31.306085999 +0000 UTC m=+24.031554521" watchObservedRunningTime="2026-07-16 04:39:31.341088502 +0000 UTC m=+24.066557023" Jul 16 04:39:31.833624 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:31.833590 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:31.833624 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:31.833614 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:31.833889 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:31.833713 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwl6r" podUID="7c288293-bb92-454b-b0de-91d7317cda4b" Jul 16 04:39:31.833889 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:31.833833 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6vzk" podUID="5da9fe69-0b36-4198-a5ff-329ddaee7389" Jul 16 04:39:32.284358 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:32.284330 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrbbv_95a3a732-50cf-47f5-9629-0b3d241d017a/ovn-acl-logging/0.log" Jul 16 04:39:32.284812 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:32.284753 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" event={"ID":"95a3a732-50cf-47f5-9629-0b3d241d017a","Type":"ContainerStarted","Data":"1308e679e909869d2b19fc157342b6a78f4cca16b4d00cde59849fc415ae1066"} Jul 16 04:39:32.287014 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:32.286983 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" event={"ID":"a4363e93-6e92-45a8-b2b7-d75a44e3fe70","Type":"ContainerStarted","Data":"9238f0b65a36dd823b341fdaf5865022a19fe8d135fa0f334d03eba89ce6ca4c"} Jul 16 04:39:32.322365 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:32.322267 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal" podStartSLOduration=23.322246045 podStartE2EDuration="23.322246045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:39:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:31.379700634 +0000 UTC m=+24.105169179" watchObservedRunningTime="2026-07-16 04:39:32.322246045 +0000 UTC m=+25.047714568" Jul 16 04:39:32.322741 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:32.322703 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vgnpx" podStartSLOduration=3.1946822360000002 podStartE2EDuration="24.322690584s" totalImagesPullingTime="21.128008348s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:39:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:32.32260628 +0000 UTC m=+25.048074819" watchObservedRunningTime="2026-07-16 04:39:32.322690584 +0000 UTC m=+25.048159107" Jul 16 04:39:32.417890 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:32.417853 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:32.594628 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:32.594551 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:32.832910 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:32.832882 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:32.833060 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:32.833011 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7g5n5" podUID="2f51d802-9618-4e88-8fc8-78b2a3788356" Jul 16 04:39:33.833455 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:33.833419 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:33.833455 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:33.833446 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:33.834051 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:33.833554 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwl6r" podUID="7c288293-bb92-454b-b0de-91d7317cda4b" Jul 16 04:39:33.834051 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:33.833631 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6vzk" podUID="5da9fe69-0b36-4198-a5ff-329ddaee7389" Jul 16 04:39:34.294432 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:34.294239 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrbbv_95a3a732-50cf-47f5-9629-0b3d241d017a/ovn-acl-logging/0.log" Jul 16 04:39:34.294952 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:34.294922 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" event={"ID":"95a3a732-50cf-47f5-9629-0b3d241d017a","Type":"ContainerStarted","Data":"abbf0f3c30810b7fd2e2a547e0288c3bfd8f05474469efca183ba14fcd7c8bd4"} Jul 16 04:39:34.295203 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:34.295172 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:34.295271 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:34.295241 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:34.295389 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:34.295372 2581 scope.go:117] "RemoveContainer" containerID="d018f4f2fcdd12a2ded930669eec40143e3a3c6466befbcc95dd157c0ef135b4" Jul 16 04:39:34.296904 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:34.296880 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9m2p" event={"ID":"99e2e193-30b8-43ed-9e81-023f10d73b10","Type":"ContainerStarted","Data":"ad2fe7ef55a16f52c53fa3868b5885e6aee15caba568fb21994e58b797f2a720"} Jul 16 04:39:34.311919 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:34.311900 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:34.833763 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:34.833682 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:34.834494 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:34.833793 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7g5n5" podUID="2f51d802-9618-4e88-8fc8-78b2a3788356" Jul 16 04:39:35.300496 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:35.300422 2581 generic.go:356] "Generic (PLEG): container finished" podID="99e2e193-30b8-43ed-9e81-023f10d73b10" containerID="ad2fe7ef55a16f52c53fa3868b5885e6aee15caba568fb21994e58b797f2a720" exitCode=0 Jul 16 04:39:35.300708 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:35.300522 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9m2p" event={"ID":"99e2e193-30b8-43ed-9e81-023f10d73b10","Type":"ContainerDied","Data":"ad2fe7ef55a16f52c53fa3868b5885e6aee15caba568fb21994e58b797f2a720"} Jul 16 04:39:35.304267 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:35.304248 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrbbv_95a3a732-50cf-47f5-9629-0b3d241d017a/ovn-acl-logging/0.log" Jul 16 04:39:35.304592 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:35.304571 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" event={"ID":"95a3a732-50cf-47f5-9629-0b3d241d017a","Type":"ContainerStarted","Data":"9f6e40b98ec15edfa1c380f5d1591714d871cabdae60f271d2d8bfe16237eb60"} Jul 16 04:39:35.304823 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:35.304804 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:35.319300 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:35.319276 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:39:35.381585 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:35.381520 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" podStartSLOduration=8.501802005 podStartE2EDuration="27.381483868s" totalImagesPullingTime="18.879681863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:35.380866845 +0000 UTC m=+28.106335384" watchObservedRunningTime="2026-07-16 04:39:35.381483868 +0000 UTC m=+28.106952390" Jul 16 04:39:35.645056 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:35.644977 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:35.732680 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:35.732653 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:35.826003 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:35.825963 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:35.833036 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:35.832926 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:35.833185 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:35.833055 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwl6r" podUID="7c288293-bb92-454b-b0de-91d7317cda4b" Jul 16 04:39:35.833377 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:35.833271 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:35.833478 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:35.833370 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6vzk" podUID="5da9fe69-0b36-4198-a5ff-329ddaee7389" Jul 16 04:39:35.937335 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:35.937251 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:36.688494 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:36.688410 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-g6vzk"] Jul 16 04:39:36.688665 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:36.688524 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:36.688665 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:36.688621 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6vzk" podUID="5da9fe69-0b36-4198-a5ff-329ddaee7389" Jul 16 04:39:36.689206 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:36.689184 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7g5n5"] Jul 16 04:39:36.689315 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:36.689273 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:36.689391 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:36.689373 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7g5n5" podUID="2f51d802-9618-4e88-8fc8-78b2a3788356" Jul 16 04:39:36.713632 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:36.713611 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bwl6r"] Jul 16 04:39:36.713695 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:36.713678 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:36.713755 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:36.713740 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwl6r" podUID="7c288293-bb92-454b-b0de-91d7317cda4b" Jul 16 04:39:36.813828 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:36.813794 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:36.934962 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:36.934924 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:37.186120 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:37.186034 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:37.294005 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:37.293972 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:37.313898 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:37.313868 2581 generic.go:356] "Generic (PLEG): container finished" podID="99e2e193-30b8-43ed-9e81-023f10d73b10" containerID="cd9df232d569cd25478347ab6323950efa6f6910aaaa123b84af03918542d349" exitCode=0 Jul 16 04:39:37.314065 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:37.313907 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9m2p" event={"ID":"99e2e193-30b8-43ed-9e81-023f10d73b10","Type":"ContainerDied","Data":"cd9df232d569cd25478347ab6323950efa6f6910aaaa123b84af03918542d349"} Jul 16 04:39:37.785392 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:37.785361 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:38.588859 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:38.588830 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:38.833553 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:38.833465 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:38.833553 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:38.833480 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:38.833553 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:38.833482 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:38.833795 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:38.833586 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwl6r" podUID="7c288293-bb92-454b-b0de-91d7317cda4b" Jul 16 04:39:38.833795 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:38.833668 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7g5n5" podUID="2f51d802-9618-4e88-8fc8-78b2a3788356" Jul 16 04:39:38.833795 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:38.833732 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6vzk" podUID="5da9fe69-0b36-4198-a5ff-329ddaee7389" Jul 16 04:39:39.319400 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:39.319315 2581 generic.go:356] "Generic (PLEG): container finished" podID="99e2e193-30b8-43ed-9e81-023f10d73b10" containerID="315125f4321ce634b5434dd038c4a63aec9ee0af06bc0cdbb6264f2dd42a437c" exitCode=0 Jul 16 04:39:39.319400 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:39.319373 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9m2p" event={"ID":"99e2e193-30b8-43ed-9e81-023f10d73b10","Type":"ContainerDied","Data":"315125f4321ce634b5434dd038c4a63aec9ee0af06bc0cdbb6264f2dd42a437c"} Jul 16 04:39:40.427169 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:40.427129 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:40.537678 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:40.537642 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:40.833726 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:40.833693 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:40.833883 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:40.833693 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:40.833883 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:40.833813 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6vzk" podUID="5da9fe69-0b36-4198-a5ff-329ddaee7389" Jul 16 04:39:40.833997 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:40.833913 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7g5n5" podUID="2f51d802-9618-4e88-8fc8-78b2a3788356" Jul 16 04:39:40.833997 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:40.833693 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:40.834095 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:40.833996 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwl6r" podUID="7c288293-bb92-454b-b0de-91d7317cda4b" Jul 16 04:39:41.621470 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:41.621435 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs\") pod \"network-metrics-daemon-7g5n5\" (UID: \"2f51d802-9618-4e88-8fc8-78b2a3788356\") " pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:41.621470 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:41.621488 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret\") pod \"global-pull-secret-syncer-g6vzk\" (UID: \"5da9fe69-0b36-4198-a5ff-329ddaee7389\") " pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:41.621968 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:41.621606 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:41.621968 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:41.621611 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:41.621968 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:41.621668 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret podName:5da9fe69-0b36-4198-a5ff-329ddaee7389 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:13.621648898 +0000 UTC m=+66.347117403 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret") pod "global-pull-secret-syncer-g6vzk" (UID: "5da9fe69-0b36-4198-a5ff-329ddaee7389") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:41.621968 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:41.621684 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs podName:2f51d802-9618-4e88-8fc8-78b2a3788356 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:13.621676771 +0000 UTC m=+66.347145270 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs") pod "network-metrics-daemon-7g5n5" (UID: "2f51d802-9618-4e88-8fc8-78b2a3788356") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:41.722216 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:41.722179 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpcdk\" (UniqueName: \"kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk\") pod \"network-check-target-bwl6r\" (UID: \"7c288293-bb92-454b-b0de-91d7317cda4b\") " pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:41.722381 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:41.722326 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:39:41.722381 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:41.722346 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:39:41.722381 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:41.722356 2581 projected.go:196] Error preparing data for projected volume kube-api-access-xpcdk for pod openshift-network-diagnostics/network-check-target-bwl6r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:41.722528 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:41.722406 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk podName:7c288293-bb92-454b-b0de-91d7317cda4b nodeName:}" failed. No retries permitted until 2026-07-16 04:40:13.722392704 +0000 UTC m=+66.447861223 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xpcdk" (UniqueName: "kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk") pod "network-check-target-bwl6r" (UID: "7c288293-bb92-454b-b0de-91d7317cda4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:42.833256 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:42.833015 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:42.833711 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:42.833021 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:42.833711 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:42.833375 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7g5n5" podUID="2f51d802-9618-4e88-8fc8-78b2a3788356" Jul 16 04:39:42.833711 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:42.833051 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:42.833711 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:42.833451 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwl6r" podUID="7c288293-bb92-454b-b0de-91d7317cda4b" Jul 16 04:39:42.833711 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:42.833566 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6vzk" podUID="5da9fe69-0b36-4198-a5ff-329ddaee7389" Jul 16 04:39:42.982314 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:42.982289 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeReady" Jul 16 04:39:42.982414 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:42.982404 2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 04:39:43.064839 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.064703 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hjvdt"] Jul 16 04:39:43.066405 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.066385 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hjvdt" Jul 16 04:39:43.070044 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.070017 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pmfl6\"" Jul 16 04:39:43.070273 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.070253 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 04:39:43.071019 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.070591 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kdmwr"] Jul 16 04:39:43.071019 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.070970 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 04:39:43.074002 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.073940 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kdmwr" Jul 16 04:39:43.077578 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.077563 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 04:39:43.078242 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.078220 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 04:39:43.078326 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.078304 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ggd6f\"" Jul 16 04:39:43.079836 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.079815 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 04:39:43.087360 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.087340 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hjvdt"] Jul 16 04:39:43.091834 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.091811 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kdmwr"] Jul 16 04:39:43.132764 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.132736 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxm26\" (UniqueName: \"kubernetes.io/projected/2f60f16a-e6e4-4289-b19b-18c9c3826d21-kube-api-access-hxm26\") pod \"ingress-canary-kdmwr\" (UID: \"2f60f16a-e6e4-4289-b19b-18c9c3826d21\") " pod="openshift-ingress-canary/ingress-canary-kdmwr" Jul 16 04:39:43.132877 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.132794 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-tmp-dir\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:39:43.132877 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.132825 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpnzb\" (UniqueName: \"kubernetes.io/projected/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-kube-api-access-jpnzb\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:39:43.132877 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.132870 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f60f16a-e6e4-4289-b19b-18c9c3826d21-cert\") pod \"ingress-canary-kdmwr\" (UID: \"2f60f16a-e6e4-4289-b19b-18c9c3826d21\") " pod="openshift-ingress-canary/ingress-canary-kdmwr" Jul 16 04:39:43.133020 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.132895 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-config-volume\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:39:43.133020 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.132931 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:39:43.233665 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.233588 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-config-volume\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:39:43.233665 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.233626 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:39:43.233820 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.233673 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxm26\" (UniqueName: \"kubernetes.io/projected/2f60f16a-e6e4-4289-b19b-18c9c3826d21-kube-api-access-hxm26\") pod \"ingress-canary-kdmwr\" (UID: \"2f60f16a-e6e4-4289-b19b-18c9c3826d21\") " pod="openshift-ingress-canary/ingress-canary-kdmwr" Jul 16 04:39:43.233820 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.233699 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-tmp-dir\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:39:43.233820 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.233719 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpnzb\" (UniqueName: \"kubernetes.io/projected/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-kube-api-access-jpnzb\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:39:43.233940 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:43.233822 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:39:43.233940 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.233890 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f60f16a-e6e4-4289-b19b-18c9c3826d21-cert\") pod \"ingress-canary-kdmwr\" (UID: \"2f60f16a-e6e4-4289-b19b-18c9c3826d21\") " pod="openshift-ingress-canary/ingress-canary-kdmwr" Jul 16 04:39:43.233940 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:43.233904 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls podName:6419d78a-657f-4e8c-8ea4-6f5a4f92c234 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:43.733882586 +0000 UTC m=+36.459351102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls") pod "dns-default-hjvdt" (UID: "6419d78a-657f-4e8c-8ea4-6f5a4f92c234") : secret "dns-default-metrics-tls" not found Jul 16 04:39:43.234133 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.234104 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-tmp-dir\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:39:43.234259 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.234240 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-config-volume\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:39:43.238010 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.237985 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f60f16a-e6e4-4289-b19b-18c9c3826d21-cert\") pod \"ingress-canary-kdmwr\" (UID: \"2f60f16a-e6e4-4289-b19b-18c9c3826d21\") " pod="openshift-ingress-canary/ingress-canary-kdmwr" Jul 16 04:39:43.252573 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.252552 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxm26\" (UniqueName: \"kubernetes.io/projected/2f60f16a-e6e4-4289-b19b-18c9c3826d21-kube-api-access-hxm26\") pod \"ingress-canary-kdmwr\" (UID: \"2f60f16a-e6e4-4289-b19b-18c9c3826d21\") " pod="openshift-ingress-canary/ingress-canary-kdmwr" Jul 16 04:39:43.258477 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.258455 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpnzb\" (UniqueName: \"kubernetes.io/projected/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-kube-api-access-jpnzb\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:39:43.332255 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.332221 2581 generic.go:356] "Generic (PLEG): container finished" podID="99e2e193-30b8-43ed-9e81-023f10d73b10" containerID="0f33eb513b1fd7807c631b18cd5e17eeb25e62f9509da2d90fa0cb53606521ec" exitCode=0 Jul 16 04:39:43.332398 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.332291 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9m2p" event={"ID":"99e2e193-30b8-43ed-9e81-023f10d73b10","Type":"ContainerDied","Data":"0f33eb513b1fd7807c631b18cd5e17eeb25e62f9509da2d90fa0cb53606521ec"} Jul 16 04:39:43.335994 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.335969 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:43.384992 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.384758 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kdmwr" Jul 16 04:39:43.475798 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.475748 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:43.539489 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.539406 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kdmwr"] Jul 16 04:39:43.543457 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:39:43.543433 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f60f16a_e6e4_4289_b19b_18c9c3826d21.slice/crio-8a64e61d7b81652cde08992c2517f0fc193f758a31ac41ad52b72e4cf39061ec WatchSource:0}: Error finding container 8a64e61d7b81652cde08992c2517f0fc193f758a31ac41ad52b72e4cf39061ec: Status 404 returned error can't find the container with id 8a64e61d7b81652cde08992c2517f0fc193f758a31ac41ad52b72e4cf39061ec Jul 16 04:39:43.604244 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.604215 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:43.736737 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:43.736689 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:39:43.736924 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:43.736845 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:39:43.736924 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:43.736907 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls podName:6419d78a-657f-4e8c-8ea4-6f5a4f92c234 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:44.736891604 +0000 UTC m=+37.462360103 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls") pod "dns-default-hjvdt" (UID: "6419d78a-657f-4e8c-8ea4-6f5a4f92c234") : secret "dns-default-metrics-tls" not found Jul 16 04:39:44.336005 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:44.335800 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kdmwr" event={"ID":"2f60f16a-e6e4-4289-b19b-18c9c3826d21","Type":"ContainerStarted","Data":"8a64e61d7b81652cde08992c2517f0fc193f758a31ac41ad52b72e4cf39061ec"} Jul 16 04:39:44.339225 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:44.339194 2581 generic.go:356] "Generic (PLEG): container finished" podID="99e2e193-30b8-43ed-9e81-023f10d73b10" containerID="7a6967cd6f3f871989a4e69145724397643dc86a8b8d76722cca9edce955f6ba" exitCode=0 Jul 16 04:39:44.339355 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:44.339257 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9m2p" event={"ID":"99e2e193-30b8-43ed-9e81-023f10d73b10","Type":"ContainerDied","Data":"7a6967cd6f3f871989a4e69145724397643dc86a8b8d76722cca9edce955f6ba"} Jul 16 04:39:44.742922 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:44.742894 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:39:44.743127 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:44.743107 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:39:44.743196 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:44.743185 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls podName:6419d78a-657f-4e8c-8ea4-6f5a4f92c234 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:46.743164342 +0000 UTC m=+39.468632846 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls") pod "dns-default-hjvdt" (UID: "6419d78a-657f-4e8c-8ea4-6f5a4f92c234") : secret "dns-default-metrics-tls" not found Jul 16 04:39:44.833395 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:44.833361 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:39:44.833557 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:44.833361 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:39:44.833557 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:44.833364 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:39:44.836354 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:44.836231 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6n555\"" Jul 16 04:39:44.836485 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:44.836438 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 04:39:44.836558 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:44.836532 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 04:39:44.837041 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:44.836788 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rjh6n\"" Jul 16 04:39:44.837188 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:44.837112 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 04:39:44.837295 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:44.837263 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 04:39:45.345624 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:45.345587 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9m2p" event={"ID":"99e2e193-30b8-43ed-9e81-023f10d73b10","Type":"ContainerStarted","Data":"ea54fcdcc9adfc5b110826889468c97292eafcb0055a409cbe559fefac9ff0be"} Jul 16 04:39:45.369797 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:45.369749 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-m9m2p" podStartSLOduration=8.85955104 podStartE2EDuration="37.369728073s" totalImagesPullingTime="28.510177033s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:39:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:45.368036597 +0000 UTC m=+38.093505143" watchObservedRunningTime="2026-07-16 04:39:45.369728073 +0000 UTC m=+38.095196595" Jul 16 04:39:45.493467 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:45.493430 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:45.635711 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:45.635623 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:46.349715 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:46.349677 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kdmwr" event={"ID":"2f60f16a-e6e4-4289-b19b-18c9c3826d21","Type":"ContainerStarted","Data":"cea06953a6cb11148a6c6cdb0f241e0a948bd990c2f22ef80ccb86dd7861d336"} Jul 16 04:39:46.369153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:46.369109 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kdmwr" podStartSLOduration=0.994273556 podStartE2EDuration="3.369093882s" totalImagesPullingTime="2.374820326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:39:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:46.368417907 +0000 UTC m=+39.093886426" watchObservedRunningTime="2026-07-16 04:39:46.369093882 +0000 UTC m=+39.094562402" Jul 16 04:39:46.756365 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:46.756338 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:39:46.756541 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:46.756481 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:39:46.756583 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:46.756564 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls podName:6419d78a-657f-4e8c-8ea4-6f5a4f92c234 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:50.756548423 +0000 UTC m=+43.482016923 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls") pod "dns-default-hjvdt" (UID: "6419d78a-657f-4e8c-8ea4-6f5a4f92c234") : secret "dns-default-metrics-tls" not found Jul 16 04:39:47.624244 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:47.624206 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:47.724307 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:47.724270 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:47.822965 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:47.822933 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:50.783051 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:50.783012 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:39:50.783497 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:50.783192 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:39:50.783497 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:50.783277 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls podName:6419d78a-657f-4e8c-8ea4-6f5a4f92c234 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:58.783254774 +0000 UTC m=+51.508723287 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls") pod "dns-default-hjvdt" (UID: "6419d78a-657f-4e8c-8ea4-6f5a4f92c234") : secret "dns-default-metrics-tls" not found Jul 16 04:39:51.706975 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:51.706936 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:51.848261 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:51.848223 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:52.557622 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:52.557586 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:52.628672 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:52.628631 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:58.837758 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:58.837715 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:39:58.838143 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:58.837871 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:39:58.838143 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:39:58.837943 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls podName:6419d78a-657f-4e8c-8ea4-6f5a4f92c234 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:14.837926523 +0000 UTC m=+67.563395028 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls") pod "dns-default-hjvdt" (UID: "6419d78a-657f-4e8c-8ea4-6f5a4f92c234") : secret "dns-default-metrics-tls" not found Jul 16 04:39:58.892682 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:58.892643 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:39:58.994056 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:39:58.994026 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:07.332987 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:07.332957 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lrbbv" Jul 16 04:40:07.650211 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:07.650124 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:07.823231 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:07.823200 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:11.776665 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:11.776626 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-cgw2x"] Jul 16 04:40:11.779319 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:11.779297 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:11.791367 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:11.791345 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 16 04:40:11.791367 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:11.791358 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 16 04:40:11.791495 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:11.791357 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 04:40:11.791902 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:11.791888 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 04:40:11.792478 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:11.792464 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-sz9bq\"" Jul 16 04:40:11.805932 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:11.805916 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 16 04:40:11.885561 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:11.885528 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-cgw2x"] Jul 16 04:40:11.921941 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:11.921913 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/aaa2dfa0-68ee-427d-86dc-bb788a81aec5-snapshots\") pod \"insights-operator-68d9dd94cf-cgw2x\" (UID: \"aaa2dfa0-68ee-427d-86dc-bb788a81aec5\") " pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:11.921941 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:11.921941 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aaa2dfa0-68ee-427d-86dc-bb788a81aec5-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-cgw2x\" (UID: \"aaa2dfa0-68ee-427d-86dc-bb788a81aec5\") " pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:11.922107 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:11.921966 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aaa2dfa0-68ee-427d-86dc-bb788a81aec5-tmp\") pod \"insights-operator-68d9dd94cf-cgw2x\" (UID: \"aaa2dfa0-68ee-427d-86dc-bb788a81aec5\") " pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:11.922107 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:11.921988 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aaa2dfa0-68ee-427d-86dc-bb788a81aec5-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-cgw2x\" (UID: \"aaa2dfa0-68ee-427d-86dc-bb788a81aec5\") " pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:11.922107 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:11.922076 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aaa2dfa0-68ee-427d-86dc-bb788a81aec5-serving-cert\") pod \"insights-operator-68d9dd94cf-cgw2x\" (UID: \"aaa2dfa0-68ee-427d-86dc-bb788a81aec5\") " pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:11.922239 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:11.922127 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hrcs\" (UniqueName: \"kubernetes.io/projected/aaa2dfa0-68ee-427d-86dc-bb788a81aec5-kube-api-access-8hrcs\") pod \"insights-operator-68d9dd94cf-cgw2x\" (UID: \"aaa2dfa0-68ee-427d-86dc-bb788a81aec5\") " pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:11.981322 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:11.981286 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v96fw"] Jul 16 04:40:11.983052 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:11.983030 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v96fw" Jul 16 04:40:11.987149 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:11.987117 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"volume-data-source-validator-78495597f6-v96fw\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-cluster-storage-operator\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" podUID="4b84faae-b7d2-45c5-b73e-4f597522ea49" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v96fw" Jul 16 04:40:11.987149 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:11.987137 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"volume-data-source-validator-dockercfg-cxb65\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-cluster-storage-operator\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-cxb65\"" type="*v1.Secret" Jul 16 04:40:11.987298 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:11.987137 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-cluster-storage-operator\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 16 04:40:11.987858 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:11.987844 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 16 04:40:12.004837 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.004819 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v96fw"] Jul 16 04:40:12.023219 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.023200 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hrcs\" (UniqueName: \"kubernetes.io/projected/aaa2dfa0-68ee-427d-86dc-bb788a81aec5-kube-api-access-8hrcs\") pod \"insights-operator-68d9dd94cf-cgw2x\" (UID: \"aaa2dfa0-68ee-427d-86dc-bb788a81aec5\") " pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:12.023304 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.023261 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/aaa2dfa0-68ee-427d-86dc-bb788a81aec5-snapshots\") pod \"insights-operator-68d9dd94cf-cgw2x\" (UID: \"aaa2dfa0-68ee-427d-86dc-bb788a81aec5\") " pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:12.023304 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.023287 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aaa2dfa0-68ee-427d-86dc-bb788a81aec5-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-cgw2x\" (UID: \"aaa2dfa0-68ee-427d-86dc-bb788a81aec5\") " pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:12.023401 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.023318 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aaa2dfa0-68ee-427d-86dc-bb788a81aec5-tmp\") pod \"insights-operator-68d9dd94cf-cgw2x\" (UID: \"aaa2dfa0-68ee-427d-86dc-bb788a81aec5\") " pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:12.023401 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.023341 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aaa2dfa0-68ee-427d-86dc-bb788a81aec5-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-cgw2x\" (UID: \"aaa2dfa0-68ee-427d-86dc-bb788a81aec5\") " pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:12.023551 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.023530 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aaa2dfa0-68ee-427d-86dc-bb788a81aec5-serving-cert\") pod \"insights-operator-68d9dd94cf-cgw2x\" (UID: \"aaa2dfa0-68ee-427d-86dc-bb788a81aec5\") " pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:12.023893 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.023876 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aaa2dfa0-68ee-427d-86dc-bb788a81aec5-tmp\") pod \"insights-operator-68d9dd94cf-cgw2x\" (UID: \"aaa2dfa0-68ee-427d-86dc-bb788a81aec5\") " pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:12.023963 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.023945 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aaa2dfa0-68ee-427d-86dc-bb788a81aec5-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-cgw2x\" (UID: \"aaa2dfa0-68ee-427d-86dc-bb788a81aec5\") " pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:12.024003 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.023952 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/aaa2dfa0-68ee-427d-86dc-bb788a81aec5-snapshots\") pod \"insights-operator-68d9dd94cf-cgw2x\" (UID: \"aaa2dfa0-68ee-427d-86dc-bb788a81aec5\") " pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:12.024339 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.024322 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aaa2dfa0-68ee-427d-86dc-bb788a81aec5-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-cgw2x\" (UID: \"aaa2dfa0-68ee-427d-86dc-bb788a81aec5\") " pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:12.027031 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.026995 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aaa2dfa0-68ee-427d-86dc-bb788a81aec5-serving-cert\") pod \"insights-operator-68d9dd94cf-cgw2x\" (UID: \"aaa2dfa0-68ee-427d-86dc-bb788a81aec5\") " pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:12.043395 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.043372 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hrcs\" (UniqueName: \"kubernetes.io/projected/aaa2dfa0-68ee-427d-86dc-bb788a81aec5-kube-api-access-8hrcs\") pod \"insights-operator-68d9dd94cf-cgw2x\" (UID: \"aaa2dfa0-68ee-427d-86dc-bb788a81aec5\") " pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:12.087250 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.087221 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" Jul 16 04:40:12.099132 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.099106 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8"] Jul 16 04:40:12.100898 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.100882 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-6zfcn"] Jul 16 04:40:12.101106 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.101087 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8" Jul 16 04:40:12.102444 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.102428 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5ddb69fbb8-rqsd6"] Jul 16 04:40:12.102617 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.102590 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-6zfcn" Jul 16 04:40:12.104015 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.103997 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.108136 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.108021 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 16 04:40:12.113881 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:12.113854 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"image-registry-tls\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" type="*v1.Secret" Jul 16 04:40:12.115106 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:12.115074 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"networking-console-plugin\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" type="*v1.ConfigMap" Jul 16 04:40:12.115987 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.115970 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 16 04:40:12.116534 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:12.116496 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"registry-dockercfg-xvsxh\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xvsxh\"" type="*v1.Secret" Jul 16 04:40:12.116593 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.116579 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 16 04:40:12.121804 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:12.121777 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"networking-console-plugin-cert\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" type="*v1.Secret" Jul 16 04:40:12.121885 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:12.121803 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"trusted-ca\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" type="*v1.ConfigMap" Jul 16 04:40:12.124193 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.124177 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dppbc\" (UniqueName: \"kubernetes.io/projected/4b84faae-b7d2-45c5-b73e-4f597522ea49-kube-api-access-dppbc\") pod \"volume-data-source-validator-78495597f6-v96fw\" (UID: \"4b84faae-b7d2-45c5-b73e-4f597522ea49\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v96fw" Jul 16 04:40:12.124724 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.124704 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 16 04:40:12.136461 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.136438 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-vt42b\"" Jul 16 04:40:12.136580 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.136527 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 04:40:12.136895 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.136877 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 04:40:12.143372 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.143352 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-bnvxv\"" Jul 16 04:40:12.167427 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.167404 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8"] Jul 16 04:40:12.188218 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.188192 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-6zfcn"] Jul 16 04:40:12.192796 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.192770 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5ddb69fbb8-rqsd6"] Jul 16 04:40:12.224788 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.224755 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6zfcn\" (UID: \"566deb82-6b60-4c93-aaeb-556a2e9bd270\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6zfcn" Jul 16 04:40:12.224788 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.224788 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.224969 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.224875 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/566deb82-6b60-4c93-aaeb-556a2e9bd270-nginx-conf\") pod \"networking-console-plugin-6cf856758c-6zfcn\" (UID: \"566deb82-6b60-4c93-aaeb-556a2e9bd270\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6zfcn" Jul 16 04:40:12.224969 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.224899 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c75845e3-f9c9-43e4-8ea1-e07a3671601a-ca-trust-extracted\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.224969 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.224937 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr26k\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-kube-api-access-lr26k\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.224969 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.224960 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dppbc\" (UniqueName: \"kubernetes.io/projected/4b84faae-b7d2-45c5-b73e-4f597522ea49-kube-api-access-dppbc\") pod \"volume-data-source-validator-78495597f6-v96fw\" (UID: \"4b84faae-b7d2-45c5-b73e-4f597522ea49\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v96fw" Jul 16 04:40:12.225105 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.225001 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c75845e3-f9c9-43e4-8ea1-e07a3671601a-image-registry-private-configuration\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.225105 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.225028 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-bound-sa-token\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.225105 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.225063 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlwj8\" (UniqueName: \"kubernetes.io/projected/fb7574c6-5d40-40ed-b909-0abde38d01ba-kube-api-access-tlwj8\") pod \"kube-storage-version-migrator-operator-799cb7f958-6vqq8\" (UID: \"fb7574c6-5d40-40ed-b909-0abde38d01ba\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8" Jul 16 04:40:12.225105 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.225092 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c75845e3-f9c9-43e4-8ea1-e07a3671601a-trusted-ca\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.225240 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.225143 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb7574c6-5d40-40ed-b909-0abde38d01ba-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-6vqq8\" (UID: \"fb7574c6-5d40-40ed-b909-0abde38d01ba\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8" Jul 16 04:40:12.225240 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.225158 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb7574c6-5d40-40ed-b909-0abde38d01ba-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-6vqq8\" (UID: \"fb7574c6-5d40-40ed-b909-0abde38d01ba\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8" Jul 16 04:40:12.225240 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.225214 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-certificates\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.225240 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.225231 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c75845e3-f9c9-43e4-8ea1-e07a3671601a-installation-pull-secrets\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.258231 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:40:12.258198 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaa2dfa0_68ee_427d_86dc_bb788a81aec5.slice/crio-846dc3af94c2e77d4357720c65fe342d8916291bb652a7d9a017132b0c51167e WatchSource:0}: Error finding container 846dc3af94c2e77d4357720c65fe342d8916291bb652a7d9a017132b0c51167e: Status 404 returned error can't find the container with id 846dc3af94c2e77d4357720c65fe342d8916291bb652a7d9a017132b0c51167e Jul 16 04:40:12.259514 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.259480 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-cgw2x"] Jul 16 04:40:12.326563 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.326464 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c75845e3-f9c9-43e4-8ea1-e07a3671601a-trusted-ca\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.326563 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.326538 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb7574c6-5d40-40ed-b909-0abde38d01ba-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-6vqq8\" (UID: \"fb7574c6-5d40-40ed-b909-0abde38d01ba\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8" Jul 16 04:40:12.326563 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.326562 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb7574c6-5d40-40ed-b909-0abde38d01ba-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-6vqq8\" (UID: \"fb7574c6-5d40-40ed-b909-0abde38d01ba\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8" Jul 16 04:40:12.326804 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.326611 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-certificates\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.326804 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.326634 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c75845e3-f9c9-43e4-8ea1-e07a3671601a-installation-pull-secrets\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.326804 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.326672 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6zfcn\" (UID: \"566deb82-6b60-4c93-aaeb-556a2e9bd270\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6zfcn" Jul 16 04:40:12.326804 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.326726 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.326985 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.326804 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/566deb82-6b60-4c93-aaeb-556a2e9bd270-nginx-conf\") pod \"networking-console-plugin-6cf856758c-6zfcn\" (UID: \"566deb82-6b60-4c93-aaeb-556a2e9bd270\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6zfcn" Jul 16 04:40:12.326985 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.326829 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c75845e3-f9c9-43e4-8ea1-e07a3671601a-ca-trust-extracted\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.326985 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.326860 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr26k\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-kube-api-access-lr26k\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.326985 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.326901 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c75845e3-f9c9-43e4-8ea1-e07a3671601a-image-registry-private-configuration\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.326985 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.326927 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-bound-sa-token\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.326985 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.326954 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlwj8\" (UniqueName: \"kubernetes.io/projected/fb7574c6-5d40-40ed-b909-0abde38d01ba-kube-api-access-tlwj8\") pod \"kube-storage-version-migrator-operator-799cb7f958-6vqq8\" (UID: \"fb7574c6-5d40-40ed-b909-0abde38d01ba\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8" Jul 16 04:40:12.327269 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.327226 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c75845e3-f9c9-43e4-8ea1-e07a3671601a-ca-trust-extracted\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.327324 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.327298 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb7574c6-5d40-40ed-b909-0abde38d01ba-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-6vqq8\" (UID: \"fb7574c6-5d40-40ed-b909-0abde38d01ba\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8" Jul 16 04:40:12.327565 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.327547 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-certificates\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.328806 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.328790 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb7574c6-5d40-40ed-b909-0abde38d01ba-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-6vqq8\" (UID: \"fb7574c6-5d40-40ed-b909-0abde38d01ba\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8" Jul 16 04:40:12.329347 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.329331 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c75845e3-f9c9-43e4-8ea1-e07a3671601a-installation-pull-secrets\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.329390 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.329364 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c75845e3-f9c9-43e4-8ea1-e07a3671601a-image-registry-private-configuration\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.342027 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.341999 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-bound-sa-token\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.343734 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.343716 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlwj8\" (UniqueName: \"kubernetes.io/projected/fb7574c6-5d40-40ed-b909-0abde38d01ba-kube-api-access-tlwj8\") pod \"kube-storage-version-migrator-operator-799cb7f958-6vqq8\" (UID: \"fb7574c6-5d40-40ed-b909-0abde38d01ba\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8" Jul 16 04:40:12.345644 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.345628 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr26k\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-kube-api-access-lr26k\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:12.411461 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.411440 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8" Jul 16 04:40:12.418046 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.418027 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" event={"ID":"aaa2dfa0-68ee-427d-86dc-bb788a81aec5","Type":"ContainerStarted","Data":"846dc3af94c2e77d4357720c65fe342d8916291bb652a7d9a017132b0c51167e"} Jul 16 04:40:12.493343 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.493309 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:12.564332 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.564305 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8"] Jul 16 04:40:12.567177 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:40:12.567150 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb7574c6_5d40_40ed_b909_0abde38d01ba.slice/crio-26bae519fcea2c9b8342dd1f0be59cffc50d553523f1eac849b6b580216389ce WatchSource:0}: Error finding container 26bae519fcea2c9b8342dd1f0be59cffc50d553523f1eac849b6b580216389ce: Status 404 returned error can't find the container with id 26bae519fcea2c9b8342dd1f0be59cffc50d553523f1eac849b6b580216389ce Jul 16 04:40:12.600580 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:12.600526 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:13.085642 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.085613 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-cxb65\"" Jul 16 04:40:13.092493 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.092474 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xvsxh\"" Jul 16 04:40:13.171540 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.171492 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 16 04:40:13.176828 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:13.176800 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 04:40:13.176965 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:13.176873 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert podName:566deb82-6b60-4c93-aaeb-556a2e9bd270 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:13.676853144 +0000 UTC m=+66.402321660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6zfcn" (UID: "566deb82-6b60-4c93-aaeb-556a2e9bd270") : secret "networking-console-plugin-cert" not found Jul 16 04:40:13.181516 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.181479 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 16 04:40:13.187931 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.187910 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/566deb82-6b60-4c93-aaeb-556a2e9bd270-nginx-conf\") pod \"networking-console-plugin-6cf856758c-6zfcn\" (UID: \"566deb82-6b60-4c93-aaeb-556a2e9bd270\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6zfcn" Jul 16 04:40:13.260781 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:13.260753 2581 projected.go:291] Couldn't get configMap openshift-cluster-storage-operator/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Jul 16 04:40:13.260944 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:13.260793 2581 projected.go:196] Error preparing data for projected volume kube-api-access-dppbc for pod openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v96fw: failed to sync configmap cache: timed out waiting for the condition Jul 16 04:40:13.260944 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:13.260864 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b84faae-b7d2-45c5-b73e-4f597522ea49-kube-api-access-dppbc podName:4b84faae-b7d2-45c5-b73e-4f597522ea49 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:13.760841712 +0000 UTC m=+66.486310231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dppbc" (UniqueName: "kubernetes.io/projected/4b84faae-b7d2-45c5-b73e-4f597522ea49-kube-api-access-dppbc") pod "volume-data-source-validator-78495597f6-v96fw" (UID: "4b84faae-b7d2-45c5-b73e-4f597522ea49") : failed to sync configmap cache: timed out waiting for the condition Jul 16 04:40:13.282043 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.282017 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 16 04:40:13.326774 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:13.326742 2581 configmap.go:193] Couldn't get configMap openshift-image-registry/trusted-ca: failed to sync configmap cache: timed out waiting for the condition Jul 16 04:40:13.326889 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:13.326831 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c75845e3-f9c9-43e4-8ea1-e07a3671601a-trusted-ca podName:c75845e3-f9c9-43e4-8ea1-e07a3671601a nodeName:}" failed. No retries permitted until 2026-07-16 04:40:13.826813674 +0000 UTC m=+66.552282178 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "trusted-ca" (UniqueName: "kubernetes.io/configmap/c75845e3-f9c9-43e4-8ea1-e07a3671601a-trusted-ca") pod "image-registry-5ddb69fbb8-rqsd6" (UID: "c75845e3-f9c9-43e4-8ea1-e07a3671601a") : failed to sync configmap cache: timed out waiting for the condition Jul 16 04:40:13.326949 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:13.326916 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: failed to sync secret cache: timed out waiting for the condition Jul 16 04:40:13.326949 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:13.326937 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5ddb69fbb8-rqsd6: failed to sync secret cache: timed out waiting for the condition Jul 16 04:40:13.327020 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:13.327000 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls podName:c75845e3-f9c9-43e4-8ea1-e07a3671601a nodeName:}" failed. No retries permitted until 2026-07-16 04:40:13.826982893 +0000 UTC m=+66.552451394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls") pod "image-registry-5ddb69fbb8-rqsd6" (UID: "c75845e3-f9c9-43e4-8ea1-e07a3671601a") : failed to sync secret cache: timed out waiting for the condition Jul 16 04:40:13.422523 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.422419 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8" event={"ID":"fb7574c6-5d40-40ed-b909-0abde38d01ba","Type":"ContainerStarted","Data":"26bae519fcea2c9b8342dd1f0be59cffc50d553523f1eac849b6b580216389ce"} Jul 16 04:40:13.533938 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.533903 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 04:40:13.636177 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.636145 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs\") pod \"network-metrics-daemon-7g5n5\" (UID: \"2f51d802-9618-4e88-8fc8-78b2a3788356\") " pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:40:13.636347 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.636222 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret\") pod \"global-pull-secret-syncer-g6vzk\" (UID: \"5da9fe69-0b36-4198-a5ff-329ddaee7389\") " pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:40:13.639321 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.639222 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 04:40:13.639437 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.639355 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 04:40:13.646555 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:13.646531 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 04:40:13.646666 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:13.646597 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs podName:2f51d802-9618-4e88-8fc8-78b2a3788356 nodeName:}" failed. No retries permitted until 2026-07-16 04:41:17.646577415 +0000 UTC m=+130.372045914 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs") pod "network-metrics-daemon-7g5n5" (UID: "2f51d802-9618-4e88-8fc8-78b2a3788356") : secret "metrics-daemon-secret" not found Jul 16 04:40:13.649117 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.649091 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5da9fe69-0b36-4198-a5ff-329ddaee7389-original-pull-secret\") pod \"global-pull-secret-syncer-g6vzk\" (UID: \"5da9fe69-0b36-4198-a5ff-329ddaee7389\") " pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:40:13.655159 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.655139 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6vzk" Jul 16 04:40:13.677175 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.677108 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 04:40:13.679430 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.679400 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:13.736618 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.736586 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpcdk\" (UniqueName: \"kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk\") pod \"network-check-target-bwl6r\" (UID: \"7c288293-bb92-454b-b0de-91d7317cda4b\") " pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:40:13.736790 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.736642 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6zfcn\" (UID: \"566deb82-6b60-4c93-aaeb-556a2e9bd270\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6zfcn" Jul 16 04:40:13.736854 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:13.736836 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 04:40:13.736927 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:13.736914 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert podName:566deb82-6b60-4c93-aaeb-556a2e9bd270 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:14.736893217 +0000 UTC m=+67.462361731 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6zfcn" (UID: "566deb82-6b60-4c93-aaeb-556a2e9bd270") : secret "networking-console-plugin-cert" not found Jul 16 04:40:13.740704 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.740664 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 04:40:13.749347 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.749320 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 04:40:13.760723 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.760690 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpcdk\" (UniqueName: \"kubernetes.io/projected/7c288293-bb92-454b-b0de-91d7317cda4b-kube-api-access-xpcdk\") pod \"network-check-target-bwl6r\" (UID: \"7c288293-bb92-454b-b0de-91d7317cda4b\") " pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:40:13.780577 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.780486 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:13.837117 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.837085 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dppbc\" (UniqueName: \"kubernetes.io/projected/4b84faae-b7d2-45c5-b73e-4f597522ea49-kube-api-access-dppbc\") pod \"volume-data-source-validator-78495597f6-v96fw\" (UID: \"4b84faae-b7d2-45c5-b73e-4f597522ea49\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v96fw" Jul 16 04:40:13.837283 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.837144 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c75845e3-f9c9-43e4-8ea1-e07a3671601a-trusted-ca\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:13.837283 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.837240 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:13.837397 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:13.837344 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 04:40:13.837397 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:13.837358 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5ddb69fbb8-rqsd6: secret "image-registry-tls" not found Jul 16 04:40:13.837494 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:13.837412 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls podName:c75845e3-f9c9-43e4-8ea1-e07a3671601a nodeName:}" failed. No retries permitted until 2026-07-16 04:40:14.837393147 +0000 UTC m=+67.562861646 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls") pod "image-registry-5ddb69fbb8-rqsd6" (UID: "c75845e3-f9c9-43e4-8ea1-e07a3671601a") : secret "image-registry-tls" not found Jul 16 04:40:13.838139 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.838111 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c75845e3-f9c9-43e4-8ea1-e07a3671601a-trusted-ca\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:13.840428 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.840387 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dppbc\" (UniqueName: \"kubernetes.io/projected/4b84faae-b7d2-45c5-b73e-4f597522ea49-kube-api-access-dppbc\") pod \"volume-data-source-validator-78495597f6-v96fw\" (UID: \"4b84faae-b7d2-45c5-b73e-4f597522ea49\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v96fw" Jul 16 04:40:13.953901 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.953822 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rjh6n\"" Jul 16 04:40:13.961143 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:13.961024 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:40:14.091334 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:14.091266 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v96fw" Jul 16 04:40:14.367628 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:14.367607 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v96fw"] Jul 16 04:40:14.385493 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:14.385464 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-g6vzk"] Jul 16 04:40:14.396489 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:14.396467 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bwl6r"] Jul 16 04:40:14.531605 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:40:14.531529 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b84faae_b7d2_45c5_b73e_4f597522ea49.slice/crio-cdd8e21f36a22ace51d82cc435997b46a2b655d918bf2b6f385abd78ec7598f8 WatchSource:0}: Error finding container cdd8e21f36a22ace51d82cc435997b46a2b655d918bf2b6f385abd78ec7598f8: Status 404 returned error can't find the container with id cdd8e21f36a22ace51d82cc435997b46a2b655d918bf2b6f385abd78ec7598f8 Jul 16 04:40:14.532425 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:40:14.532400 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5da9fe69_0b36_4198_a5ff_329ddaee7389.slice/crio-860e9aa9fbd22a9f69e52ecf95d524e8ddaa5d87c2ad3beca0d2445a6c165a56 WatchSource:0}: Error finding container 860e9aa9fbd22a9f69e52ecf95d524e8ddaa5d87c2ad3beca0d2445a6c165a56: Status 404 returned error can't find the container with id 860e9aa9fbd22a9f69e52ecf95d524e8ddaa5d87c2ad3beca0d2445a6c165a56 Jul 16 04:40:14.532714 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:40:14.532694 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c288293_bb92_454b_b0de_91d7317cda4b.slice/crio-ae4992b25ea2e4793c4a2e84fb15dc7cea7d117579b6bfcd7aab2e26b33e83e3 WatchSource:0}: Error finding container ae4992b25ea2e4793c4a2e84fb15dc7cea7d117579b6bfcd7aab2e26b33e83e3: Status 404 returned error can't find the container with id ae4992b25ea2e4793c4a2e84fb15dc7cea7d117579b6bfcd7aab2e26b33e83e3 Jul 16 04:40:14.745752 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:14.745717 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6zfcn\" (UID: \"566deb82-6b60-4c93-aaeb-556a2e9bd270\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6zfcn" Jul 16 04:40:14.745960 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:14.745883 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 04:40:14.746012 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:14.745967 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert podName:566deb82-6b60-4c93-aaeb-556a2e9bd270 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:16.74594619 +0000 UTC m=+69.471414692 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6zfcn" (UID: "566deb82-6b60-4c93-aaeb-556a2e9bd270") : secret "networking-console-plugin-cert" not found Jul 16 04:40:14.847189 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:14.847105 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:40:14.847337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:14.847190 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:14.847337 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:14.847269 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:40:14.847337 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:14.847304 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 04:40:14.847337 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:14.847317 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5ddb69fbb8-rqsd6: secret "image-registry-tls" not found Jul 16 04:40:14.847337 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:14.847330 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls podName:6419d78a-657f-4e8c-8ea4-6f5a4f92c234 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:46.847316175 +0000 UTC m=+99.572784674 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls") pod "dns-default-hjvdt" (UID: "6419d78a-657f-4e8c-8ea4-6f5a4f92c234") : secret "dns-default-metrics-tls" not found Jul 16 04:40:14.847556 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:14.847362 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls podName:c75845e3-f9c9-43e4-8ea1-e07a3671601a nodeName:}" failed. No retries permitted until 2026-07-16 04:40:16.847347468 +0000 UTC m=+69.572815971 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls") pod "image-registry-5ddb69fbb8-rqsd6" (UID: "c75845e3-f9c9-43e4-8ea1-e07a3671601a") : secret "image-registry-tls" not found Jul 16 04:40:15.427925 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:15.427883 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bwl6r" event={"ID":"7c288293-bb92-454b-b0de-91d7317cda4b","Type":"ContainerStarted","Data":"ae4992b25ea2e4793c4a2e84fb15dc7cea7d117579b6bfcd7aab2e26b33e83e3"} Jul 16 04:40:15.429154 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:15.429128 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v96fw" event={"ID":"4b84faae-b7d2-45c5-b73e-4f597522ea49","Type":"ContainerStarted","Data":"cdd8e21f36a22ace51d82cc435997b46a2b655d918bf2b6f385abd78ec7598f8"} Jul 16 04:40:15.431355 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:15.431304 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" event={"ID":"aaa2dfa0-68ee-427d-86dc-bb788a81aec5","Type":"ContainerStarted","Data":"f3a5be60417895213c242b60324d75f6bc15c74b7b45d70a328d1fb181efeede"} Jul 16 04:40:15.432982 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:15.432958 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-g6vzk" event={"ID":"5da9fe69-0b36-4198-a5ff-329ddaee7389","Type":"ContainerStarted","Data":"860e9aa9fbd22a9f69e52ecf95d524e8ddaa5d87c2ad3beca0d2445a6c165a56"} Jul 16 04:40:16.439576 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:16.439532 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8" event={"ID":"fb7574c6-5d40-40ed-b909-0abde38d01ba","Type":"ContainerStarted","Data":"0f15e6a83987c86e1c3d92c4bb9ec2861c61692dacd960bcc1074a2a0d01a8cf"} Jul 16 04:40:16.465244 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:16.465142 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" podStartSLOduration=2.674885398 podStartE2EDuration="5.465122512s" totalImagesPullingTime="2.790237114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:15.463235087 +0000 UTC m=+68.188703609" watchObservedRunningTime="2026-07-16 04:40:16.465122512 +0000 UTC m=+69.190591029" Jul 16 04:40:16.509226 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:16.509191 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:16.636649 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:16.636614 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:16.766034 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:16.765950 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6zfcn\" (UID: \"566deb82-6b60-4c93-aaeb-556a2e9bd270\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6zfcn" Jul 16 04:40:16.766248 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:16.766120 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 04:40:16.766248 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:16.766210 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert podName:566deb82-6b60-4c93-aaeb-556a2e9bd270 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:20.766190664 +0000 UTC m=+73.491659178 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6zfcn" (UID: "566deb82-6b60-4c93-aaeb-556a2e9bd270") : secret "networking-console-plugin-cert" not found Jul 16 04:40:16.867260 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:16.866695 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:16.867260 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:16.866857 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 04:40:16.867260 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:16.866872 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5ddb69fbb8-rqsd6: secret "image-registry-tls" not found Jul 16 04:40:16.867260 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:16.866927 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls podName:c75845e3-f9c9-43e4-8ea1-e07a3671601a nodeName:}" failed. No retries permitted until 2026-07-16 04:40:20.866909944 +0000 UTC m=+73.592378447 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls") pod "image-registry-5ddb69fbb8-rqsd6" (UID: "c75845e3-f9c9-43e4-8ea1-e07a3671601a") : secret "image-registry-tls" not found Jul 16 04:40:16.974992 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:16.974933 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8" podStartSLOduration=2.138461455 podStartE2EDuration="4.974896432s" totalImagesPullingTime="2.836434977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:16.464837582 +0000 UTC m=+69.190306127" watchObservedRunningTime="2026-07-16 04:40:16.974896432 +0000 UTC m=+69.700364954" Jul 16 04:40:16.976170 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:16.976053 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-w92t7"] Jul 16 04:40:16.978711 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:16.978476 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:16.985798 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:16.985542 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 04:40:16.985920 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:16.985852 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 04:40:16.989215 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:16.989190 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-68dzs\"" Jul 16 04:40:17.014427 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.014388 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w92t7"] Jul 16 04:40:17.036196 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.036057 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:17.068614 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.068579 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/794de458-b940-478f-a21b-9d2450761e3d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:17.068614 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.068620 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/794de458-b940-478f-a21b-9d2450761e3d-crio-socket\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:17.068848 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.068759 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfrbn\" (UniqueName: \"kubernetes.io/projected/794de458-b940-478f-a21b-9d2450761e3d-kube-api-access-vfrbn\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:17.068848 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.068798 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/794de458-b940-478f-a21b-9d2450761e3d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:17.068848 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.068830 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/794de458-b940-478f-a21b-9d2450761e3d-data-volume\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:17.170298 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.170255 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/794de458-b940-478f-a21b-9d2450761e3d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:17.170298 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.170311 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/794de458-b940-478f-a21b-9d2450761e3d-crio-socket\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:17.170576 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.170377 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfrbn\" (UniqueName: \"kubernetes.io/projected/794de458-b940-478f-a21b-9d2450761e3d-kube-api-access-vfrbn\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:17.170576 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.170406 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/794de458-b940-478f-a21b-9d2450761e3d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:17.170576 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:17.170410 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 04:40:17.170576 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.170433 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/794de458-b940-478f-a21b-9d2450761e3d-data-volume\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:17.170576 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:17.170494 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/794de458-b940-478f-a21b-9d2450761e3d-kube-rbac-proxy-cm podName:794de458-b940-478f-a21b-9d2450761e3d nodeName:}" failed. No retries permitted until 2026-07-16 04:40:17.670472144 +0000 UTC m=+70.395940645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/794de458-b940-478f-a21b-9d2450761e3d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w92t7" (UID: "794de458-b940-478f-a21b-9d2450761e3d") : configmap "kube-rbac-proxy" not found Jul 16 04:40:17.170836 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.170776 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/794de458-b940-478f-a21b-9d2450761e3d-data-volume\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:17.170896 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.170844 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/794de458-b940-478f-a21b-9d2450761e3d-crio-socket\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:17.170952 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:17.170922 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 04:40:17.171001 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:17.170976 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/794de458-b940-478f-a21b-9d2450761e3d-insights-runtime-extractor-tls podName:794de458-b940-478f-a21b-9d2450761e3d nodeName:}" failed. No retries permitted until 2026-07-16 04:40:17.670962595 +0000 UTC m=+70.396431101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/794de458-b940-478f-a21b-9d2450761e3d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w92t7" (UID: "794de458-b940-478f-a21b-9d2450761e3d") : secret "insights-runtime-extractor-tls" not found Jul 16 04:40:17.214128 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.214062 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfrbn\" (UniqueName: \"kubernetes.io/projected/794de458-b940-478f-a21b-9d2450761e3d-kube-api-access-vfrbn\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:17.296633 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.296553 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:17.436466 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.436428 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:17.443913 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.443881 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v96fw" event={"ID":"4b84faae-b7d2-45c5-b73e-4f597522ea49","Type":"ContainerStarted","Data":"f826f9f56362bf292954cefdcb9de0565cdddbbbce9ada4cca352297fe8e6228"} Jul 16 04:40:17.474135 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.473785 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-v96fw" podStartSLOduration=4.433666097 podStartE2EDuration="6.473767352s" totalImagesPullingTime="2.040101255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:17.471719402 +0000 UTC m=+70.197187923" watchObservedRunningTime="2026-07-16 04:40:17.473767352 +0000 UTC m=+70.199235874" Jul 16 04:40:17.662639 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.662434 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:17.675842 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.675807 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/794de458-b940-478f-a21b-9d2450761e3d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:17.676134 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:17.675982 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 04:40:17.676134 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.675999 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/794de458-b940-478f-a21b-9d2450761e3d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:17.676134 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:17.676053 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/794de458-b940-478f-a21b-9d2450761e3d-insights-runtime-extractor-tls podName:794de458-b940-478f-a21b-9d2450761e3d nodeName:}" failed. No retries permitted until 2026-07-16 04:40:18.676033885 +0000 UTC m=+71.401502386 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/794de458-b940-478f-a21b-9d2450761e3d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w92t7" (UID: "794de458-b940-478f-a21b-9d2450761e3d") : secret "insights-runtime-extractor-tls" not found Jul 16 04:40:17.676134 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:17.676101 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 04:40:17.676417 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:17.676162 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/794de458-b940-478f-a21b-9d2450761e3d-kube-rbac-proxy-cm podName:794de458-b940-478f-a21b-9d2450761e3d nodeName:}" failed. No retries permitted until 2026-07-16 04:40:18.676146528 +0000 UTC m=+71.401615029 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/794de458-b940-478f-a21b-9d2450761e3d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w92t7" (UID: "794de458-b940-478f-a21b-9d2450761e3d") : configmap "kube-rbac-proxy" not found Jul 16 04:40:17.811201 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:17.811162 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:18.539383 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:18.539327 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:18.650553 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:18.650494 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:18.684593 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:18.684557 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/794de458-b940-478f-a21b-9d2450761e3d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:18.684772 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:18.684654 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/794de458-b940-478f-a21b-9d2450761e3d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:18.684772 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:18.684703 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 04:40:18.684772 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:18.684769 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 04:40:18.684927 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:18.684782 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/794de458-b940-478f-a21b-9d2450761e3d-kube-rbac-proxy-cm podName:794de458-b940-478f-a21b-9d2450761e3d nodeName:}" failed. No retries permitted until 2026-07-16 04:40:20.684761945 +0000 UTC m=+73.410230447 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/794de458-b940-478f-a21b-9d2450761e3d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w92t7" (UID: "794de458-b940-478f-a21b-9d2450761e3d") : configmap "kube-rbac-proxy" not found Jul 16 04:40:18.684927 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:18.684806 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/794de458-b940-478f-a21b-9d2450761e3d-insights-runtime-extractor-tls podName:794de458-b940-478f-a21b-9d2450761e3d nodeName:}" failed. No retries permitted until 2026-07-16 04:40:20.684796331 +0000 UTC m=+73.410264842 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/794de458-b940-478f-a21b-9d2450761e3d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w92t7" (UID: "794de458-b940-478f-a21b-9d2450761e3d") : secret "insights-runtime-extractor-tls" not found Jul 16 04:40:18.844632 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:18.844546 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-p6f24"] Jul 16 04:40:18.846725 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:18.846702 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-p6f24" Jul 16 04:40:18.850337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:18.850315 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 16 04:40:18.850850 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:18.850831 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 16 04:40:18.851299 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:18.851277 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 16 04:40:18.851393 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:18.851277 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-69b2s\"" Jul 16 04:40:18.851393 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:18.851277 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 16 04:40:18.869101 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:18.869060 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-p6f24"] Jul 16 04:40:18.925279 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:18.925231 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:18.986890 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:18.986860 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf66m\" (UniqueName: \"kubernetes.io/projected/d270498d-612f-4e42-9bf8-df2a7f04712a-kube-api-access-zf66m\") pod \"service-ca-79f68b9f4f-p6f24\" (UID: \"d270498d-612f-4e42-9bf8-df2a7f04712a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-p6f24" Jul 16 04:40:18.987068 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:18.986998 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d270498d-612f-4e42-9bf8-df2a7f04712a-signing-key\") pod \"service-ca-79f68b9f4f-p6f24\" (UID: \"d270498d-612f-4e42-9bf8-df2a7f04712a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-p6f24" Jul 16 04:40:18.987068 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:18.987056 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d270498d-612f-4e42-9bf8-df2a7f04712a-signing-cabundle\") pod \"service-ca-79f68b9f4f-p6f24\" (UID: \"d270498d-612f-4e42-9bf8-df2a7f04712a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-p6f24" Jul 16 04:40:19.032518 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:19.032483 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:19.087761 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:19.087728 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d270498d-612f-4e42-9bf8-df2a7f04712a-signing-key\") pod \"service-ca-79f68b9f4f-p6f24\" (UID: \"d270498d-612f-4e42-9bf8-df2a7f04712a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-p6f24" Jul 16 04:40:19.087925 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:19.087774 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d270498d-612f-4e42-9bf8-df2a7f04712a-signing-cabundle\") pod \"service-ca-79f68b9f4f-p6f24\" (UID: \"d270498d-612f-4e42-9bf8-df2a7f04712a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-p6f24" Jul 16 04:40:19.087925 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:19.087856 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf66m\" (UniqueName: \"kubernetes.io/projected/d270498d-612f-4e42-9bf8-df2a7f04712a-kube-api-access-zf66m\") pod \"service-ca-79f68b9f4f-p6f24\" (UID: \"d270498d-612f-4e42-9bf8-df2a7f04712a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-p6f24" Jul 16 04:40:19.088676 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:19.088649 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d270498d-612f-4e42-9bf8-df2a7f04712a-signing-cabundle\") pod \"service-ca-79f68b9f4f-p6f24\" (UID: \"d270498d-612f-4e42-9bf8-df2a7f04712a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-p6f24" Jul 16 04:40:19.090259 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:19.090237 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d270498d-612f-4e42-9bf8-df2a7f04712a-signing-key\") pod \"service-ca-79f68b9f4f-p6f24\" (UID: \"d270498d-612f-4e42-9bf8-df2a7f04712a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-p6f24" Jul 16 04:40:19.099322 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:19.099271 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf66m\" (UniqueName: \"kubernetes.io/projected/d270498d-612f-4e42-9bf8-df2a7f04712a-kube-api-access-zf66m\") pod \"service-ca-79f68b9f4f-p6f24\" (UID: \"d270498d-612f-4e42-9bf8-df2a7f04712a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-p6f24" Jul 16 04:40:19.100224 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:19.100206 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xjxgr_01e9bde7-f187-4fe4-b933-e31dfc8de92b/dns-node-resolver/0.log" Jul 16 04:40:19.157726 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:19.157693 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-p6f24" Jul 16 04:40:19.754981 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:19.754942 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-p6f24"] Jul 16 04:40:19.758680 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:40:19.758654 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd270498d_612f_4e42_9bf8_df2a7f04712a.slice/crio-51d2c7021c7b938d1b90561d554e791f81adf5422900241715a145125e79e02d WatchSource:0}: Error finding container 51d2c7021c7b938d1b90561d554e791f81adf5422900241715a145125e79e02d: Status 404 returned error can't find the container with id 51d2c7021c7b938d1b90561d554e791f81adf5422900241715a145125e79e02d Jul 16 04:40:19.904303 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:19.904236 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cngtn_852b5867-4340-4c84-9da8-8e4f02270f0d/node-ca/0.log" Jul 16 04:40:20.334801 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:20.334761 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:20.454727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:20.454688 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bwl6r" event={"ID":"7c288293-bb92-454b-b0de-91d7317cda4b","Type":"ContainerStarted","Data":"bf8d37f99491a313e42252e650942611e1213f0a9e6640129ef0a2e45dcb2448"} Jul 16 04:40:20.454897 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:20.454798 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:40:20.456567 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:20.456529 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-g6vzk" event={"ID":"5da9fe69-0b36-4198-a5ff-329ddaee7389","Type":"ContainerStarted","Data":"6911431c91af85275fd9dffecdb443ac5d51b391620a1c8ba33d218ed4219529"} Jul 16 04:40:20.457698 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:20.457675 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-p6f24" event={"ID":"d270498d-612f-4e42-9bf8-df2a7f04712a","Type":"ContainerStarted","Data":"51d2c7021c7b938d1b90561d554e791f81adf5422900241715a145125e79e02d"} Jul 16 04:40:20.490065 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:20.489997 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bwl6r" podStartSLOduration=67.432634644 podStartE2EDuration="1m12.489978759s" totalImagesPullingTime="5.057344115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:20.489540731 +0000 UTC m=+73.215009252" watchObservedRunningTime="2026-07-16 04:40:20.489978759 +0000 UTC m=+73.215447285" Jul 16 04:40:20.520257 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:20.520198 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-g6vzk" podStartSLOduration=67.455584095 podStartE2EDuration="1m12.520177562s" totalImagesPullingTime="5.064593467s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:20.518911033 +0000 UTC m=+73.244379555" watchObservedRunningTime="2026-07-16 04:40:20.520177562 +0000 UTC m=+73.245646083" Jul 16 04:40:20.526037 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:20.526010 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kdmwr_2f60f16a-e6e4-4289-b19b-18c9c3826d21/serve-healthcheck-canary/0.log" Jul 16 04:40:20.703018 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:20.702910 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/794de458-b940-478f-a21b-9d2450761e3d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:20.703018 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:20.702994 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/794de458-b940-478f-a21b-9d2450761e3d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:20.703427 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:20.703052 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 04:40:20.703427 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:20.703133 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/794de458-b940-478f-a21b-9d2450761e3d-kube-rbac-proxy-cm podName:794de458-b940-478f-a21b-9d2450761e3d nodeName:}" failed. No retries permitted until 2026-07-16 04:40:24.703111543 +0000 UTC m=+77.428580061 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/794de458-b940-478f-a21b-9d2450761e3d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w92t7" (UID: "794de458-b940-478f-a21b-9d2450761e3d") : configmap "kube-rbac-proxy" not found Jul 16 04:40:20.703427 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:20.703153 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 04:40:20.703427 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:20.703230 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/794de458-b940-478f-a21b-9d2450761e3d-insights-runtime-extractor-tls podName:794de458-b940-478f-a21b-9d2450761e3d nodeName:}" failed. No retries permitted until 2026-07-16 04:40:24.703212252 +0000 UTC m=+77.428680754 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/794de458-b940-478f-a21b-9d2450761e3d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w92t7" (UID: "794de458-b940-478f-a21b-9d2450761e3d") : secret "insights-runtime-extractor-tls" not found Jul 16 04:40:20.803402 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:20.803360 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6zfcn\" (UID: \"566deb82-6b60-4c93-aaeb-556a2e9bd270\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6zfcn" Jul 16 04:40:20.804068 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:20.803533 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 04:40:20.804068 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:20.803596 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert podName:566deb82-6b60-4c93-aaeb-556a2e9bd270 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:28.803581511 +0000 UTC m=+81.529050011 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6zfcn" (UID: "566deb82-6b60-4c93-aaeb-556a2e9bd270") : secret "networking-console-plugin-cert" not found Jul 16 04:40:20.904354 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:20.904313 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:20.904546 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:20.904477 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 04:40:20.904546 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:20.904493 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5ddb69fbb8-rqsd6: secret "image-registry-tls" not found Jul 16 04:40:20.904644 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:20.904566 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls podName:c75845e3-f9c9-43e4-8ea1-e07a3671601a nodeName:}" failed. No retries permitted until 2026-07-16 04:40:28.904547609 +0000 UTC m=+81.630016112 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls") pod "image-registry-5ddb69fbb8-rqsd6" (UID: "c75845e3-f9c9-43e4-8ea1-e07a3671601a") : secret "image-registry-tls" not found Jul 16 04:40:22.364162 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:22.364131 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:22.467637 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:22.467570 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-p6f24" event={"ID":"d270498d-612f-4e42-9bf8-df2a7f04712a","Type":"ContainerStarted","Data":"5bf1a8a785f8cc580c8a23abfad33422e94390e788814cb8d47b534d16251edd"} Jul 16 04:40:22.545790 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:22.545758 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:22.632714 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:22.632629 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:22.733401 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:22.733329 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:23.046848 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:23.046803 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:24.044125 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:24.044094 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:24.441022 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:24.440924 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:24.694272 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:24.694183 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:24.741441 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:24.741406 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/794de458-b940-478f-a21b-9d2450761e3d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:24.741617 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:24.741489 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/794de458-b940-478f-a21b-9d2450761e3d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:24.741617 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:24.741578 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 04:40:24.741617 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:24.741599 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 04:40:24.741775 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:24.741669 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/794de458-b940-478f-a21b-9d2450761e3d-insights-runtime-extractor-tls podName:794de458-b940-478f-a21b-9d2450761e3d nodeName:}" failed. No retries permitted until 2026-07-16 04:40:32.741649417 +0000 UTC m=+85.467117936 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/794de458-b940-478f-a21b-9d2450761e3d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w92t7" (UID: "794de458-b940-478f-a21b-9d2450761e3d") : secret "insights-runtime-extractor-tls" not found Jul 16 04:40:24.741775 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:24.741689 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/794de458-b940-478f-a21b-9d2450761e3d-kube-rbac-proxy-cm podName:794de458-b940-478f-a21b-9d2450761e3d nodeName:}" failed. No retries permitted until 2026-07-16 04:40:32.741680192 +0000 UTC m=+85.467148692 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/794de458-b940-478f-a21b-9d2450761e3d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w92t7" (UID: "794de458-b940-478f-a21b-9d2450761e3d") : configmap "kube-rbac-proxy" not found Jul 16 04:40:24.869175 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:24.869144 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:25.043115 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:25.043084 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:26.070187 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:26.070140 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:26.176026 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:26.175986 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:26.273441 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:26.273405 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:26.673851 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:26.673813 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:26.884889 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:26.884857 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:27.009228 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:27.009194 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:27.514307 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:27.514274 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:27.867489 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:27.867398 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:28.380033 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:28.379993 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:28.605870 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:28.605838 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:28.701193 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:28.701106 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:28.876416 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:28.876369 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6zfcn\" (UID: \"566deb82-6b60-4c93-aaeb-556a2e9bd270\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6zfcn" Jul 16 04:40:28.876614 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:28.876542 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 04:40:28.876614 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:40:28.876606 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert podName:566deb82-6b60-4c93-aaeb-556a2e9bd270 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:44.87659067 +0000 UTC m=+97.602059171 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6zfcn" (UID: "566deb82-6b60-4c93-aaeb-556a2e9bd270") : secret "networking-console-plugin-cert" not found Jul 16 04:40:28.880658 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:28.880635 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:28.977813 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:28.977785 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:28.980049 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:28.980029 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls\") pod \"image-registry-5ddb69fbb8-rqsd6\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:29.222803 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:29.222769 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:29.245886 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:29.245854 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:29.379384 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:29.377306 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-p6f24" podStartSLOduration=9.715412004000001 podStartE2EDuration="11.377279113s" totalImagesPullingTime="1.661867109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:40:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:22.505932851 +0000 UTC m=+75.231401374" watchObservedRunningTime="2026-07-16 04:40:29.377279113 +0000 UTC m=+82.102747640" Jul 16 04:40:29.379384 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:29.377801 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5ddb69fbb8-rqsd6"] Jul 16 04:40:29.382198 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:40:29.382164 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc75845e3_f9c9_43e4_8ea1_e07a3671601a.slice/crio-53e4bad4d7aafa7cbfb5a61b271a46eb1947ec73787541d3fc1c43eb6c52bcd3 WatchSource:0}: Error finding container 53e4bad4d7aafa7cbfb5a61b271a46eb1947ec73787541d3fc1c43eb6c52bcd3: Status 404 returned error can't find the container with id 53e4bad4d7aafa7cbfb5a61b271a46eb1947ec73787541d3fc1c43eb6c52bcd3 Jul 16 04:40:29.490850 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:29.490770 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" event={"ID":"c75845e3-f9c9-43e4-8ea1-e07a3671601a","Type":"ContainerStarted","Data":"f8597d2214b1b9ca9727db5e9e090589ec7b59885abef9a533847ac2912c3163"} Jul 16 04:40:29.490850 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:29.490822 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" event={"ID":"c75845e3-f9c9-43e4-8ea1-e07a3671601a","Type":"ContainerStarted","Data":"53e4bad4d7aafa7cbfb5a61b271a46eb1947ec73787541d3fc1c43eb6c52bcd3"} Jul 16 04:40:29.491020 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:29.490915 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:40:30.063896 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:30.063772 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:30.259977 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:30.259935 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:30.375876 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:30.375798 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:30.527356 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:30.527300 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:30.697702 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:30.697610 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:31.872248 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:31.872214 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:32.058778 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:32.058745 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:32.275366 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:32.275338 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:32.464416 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:32.464387 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:32.670944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:32.670865 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:32.809175 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:32.809139 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/794de458-b940-478f-a21b-9d2450761e3d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:32.809327 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:32.809215 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/794de458-b940-478f-a21b-9d2450761e3d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:32.809730 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:32.809712 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/794de458-b940-478f-a21b-9d2450761e3d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:32.811625 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:32.811605 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/794de458-b940-478f-a21b-9d2450761e3d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w92t7\" (UID: \"794de458-b940-478f-a21b-9d2450761e3d\") " pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:32.890789 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:32.890757 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w92t7" Jul 16 04:40:33.034381 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:33.034331 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" podStartSLOduration=21.034311057 podStartE2EDuration="21.034311057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:40:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:29.533560058 +0000 UTC m=+82.259028579" watchObservedRunningTime="2026-07-16 04:40:33.034311057 +0000 UTC m=+85.759779584" Jul 16 04:40:33.034720 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:33.034703 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w92t7"] Jul 16 04:40:33.037757 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:40:33.037725 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod794de458_b940_478f_a21b_9d2450761e3d.slice/crio-226574e6454b3ae6376e6b07e051d59e6949ef764d382ddea901111167b5b6c9 WatchSource:0}: Error finding container 226574e6454b3ae6376e6b07e051d59e6949ef764d382ddea901111167b5b6c9: Status 404 returned error can't find the container with id 226574e6454b3ae6376e6b07e051d59e6949ef764d382ddea901111167b5b6c9 Jul 16 04:40:33.084751 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:33.084711 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:33.507011 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:33.506972 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w92t7" event={"ID":"794de458-b940-478f-a21b-9d2450761e3d","Type":"ContainerStarted","Data":"226574e6454b3ae6376e6b07e051d59e6949ef764d382ddea901111167b5b6c9"} Jul 16 04:40:38.209383 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:38.209347 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:38.330794 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:38.330712 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:38.527258 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:38.527173 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w92t7" event={"ID":"794de458-b940-478f-a21b-9d2450761e3d","Type":"ContainerStarted","Data":"f5c3022151f9b60ac4eb36a9bd71b501a1ba21b77080cae86c528a6fc614dbd0"} Jul 16 04:40:38.607792 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:38.607660 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:38.871224 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:38.871088 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:39.073877 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:39.073837 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:39.738329 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:39.738299 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:40.123926 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:40.123836 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:40.259911 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:40.259875 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:40.524243 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:40.524211 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:40.664356 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:40.664320 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:41.538520 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:41.538463 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w92t7" event={"ID":"794de458-b940-478f-a21b-9d2450761e3d","Type":"ContainerStarted","Data":"fbcb260a718dcb300f58e5034db5f29b5fcdf6b051d4c20c4e2114a5ff1a6a1e"} Jul 16 04:40:42.326517 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.326468 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:42.827838 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.827804 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82"] Jul 16 04:40:42.831207 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.831179 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:42.835303 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.835281 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 16 04:40:42.835441 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.835312 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 16 04:40:42.835587 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.835546 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 16 04:40:42.835669 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.835605 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 16 04:40:42.835715 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.835673 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 16 04:40:42.836040 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.836021 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 16 04:40:42.836150 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.836097 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 16 04:40:42.874519 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.874469 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82"] Jul 16 04:40:42.895954 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.895887 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d264612c-c1e6-4c14-b57d-11bf19bbd320-ca\") pod \"cluster-proxy-proxy-agent-5744455dbd-b4d82\" (UID: \"d264612c-c1e6-4c14-b57d-11bf19bbd320\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:42.896173 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.895961 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d264612c-c1e6-4c14-b57d-11bf19bbd320-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5744455dbd-b4d82\" (UID: \"d264612c-c1e6-4c14-b57d-11bf19bbd320\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:42.896173 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.896054 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d264612c-c1e6-4c14-b57d-11bf19bbd320-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5744455dbd-b4d82\" (UID: \"d264612c-c1e6-4c14-b57d-11bf19bbd320\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:42.896173 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.896100 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwv48\" (UniqueName: \"kubernetes.io/projected/d264612c-c1e6-4c14-b57d-11bf19bbd320-kube-api-access-cwv48\") pod \"cluster-proxy-proxy-agent-5744455dbd-b4d82\" (UID: \"d264612c-c1e6-4c14-b57d-11bf19bbd320\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:42.896173 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.896155 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d264612c-c1e6-4c14-b57d-11bf19bbd320-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5744455dbd-b4d82\" (UID: \"d264612c-c1e6-4c14-b57d-11bf19bbd320\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:42.896432 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.896184 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d264612c-c1e6-4c14-b57d-11bf19bbd320-hub\") pod \"cluster-proxy-proxy-agent-5744455dbd-b4d82\" (UID: \"d264612c-c1e6-4c14-b57d-11bf19bbd320\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:42.937626 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.937590 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5ddb69fbb8-rqsd6"] Jul 16 04:40:42.997023 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.996992 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d264612c-c1e6-4c14-b57d-11bf19bbd320-hub\") pod \"cluster-proxy-proxy-agent-5744455dbd-b4d82\" (UID: \"d264612c-c1e6-4c14-b57d-11bf19bbd320\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:42.997246 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.997084 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d264612c-c1e6-4c14-b57d-11bf19bbd320-ca\") pod \"cluster-proxy-proxy-agent-5744455dbd-b4d82\" (UID: \"d264612c-c1e6-4c14-b57d-11bf19bbd320\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:42.997246 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.997114 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d264612c-c1e6-4c14-b57d-11bf19bbd320-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5744455dbd-b4d82\" (UID: \"d264612c-c1e6-4c14-b57d-11bf19bbd320\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:42.997246 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.997140 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d264612c-c1e6-4c14-b57d-11bf19bbd320-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5744455dbd-b4d82\" (UID: \"d264612c-c1e6-4c14-b57d-11bf19bbd320\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:42.997246 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.997165 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwv48\" (UniqueName: \"kubernetes.io/projected/d264612c-c1e6-4c14-b57d-11bf19bbd320-kube-api-access-cwv48\") pod \"cluster-proxy-proxy-agent-5744455dbd-b4d82\" (UID: \"d264612c-c1e6-4c14-b57d-11bf19bbd320\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:42.997246 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.997206 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d264612c-c1e6-4c14-b57d-11bf19bbd320-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5744455dbd-b4d82\" (UID: \"d264612c-c1e6-4c14-b57d-11bf19bbd320\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:42.998004 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.997974 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d264612c-c1e6-4c14-b57d-11bf19bbd320-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5744455dbd-b4d82\" (UID: \"d264612c-c1e6-4c14-b57d-11bf19bbd320\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:42.999950 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:42.999918 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d264612c-c1e6-4c14-b57d-11bf19bbd320-ca\") pod \"cluster-proxy-proxy-agent-5744455dbd-b4d82\" (UID: \"d264612c-c1e6-4c14-b57d-11bf19bbd320\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:43.000055 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:43.000022 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d264612c-c1e6-4c14-b57d-11bf19bbd320-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5744455dbd-b4d82\" (UID: \"d264612c-c1e6-4c14-b57d-11bf19bbd320\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:43.000190 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:43.000161 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d264612c-c1e6-4c14-b57d-11bf19bbd320-hub\") pod \"cluster-proxy-proxy-agent-5744455dbd-b4d82\" (UID: \"d264612c-c1e6-4c14-b57d-11bf19bbd320\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:43.000350 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:43.000331 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d264612c-c1e6-4c14-b57d-11bf19bbd320-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5744455dbd-b4d82\" (UID: \"d264612c-c1e6-4c14-b57d-11bf19bbd320\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:43.023035 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:43.023004 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwv48\" (UniqueName: \"kubernetes.io/projected/d264612c-c1e6-4c14-b57d-11bf19bbd320-kube-api-access-cwv48\") pod \"cluster-proxy-proxy-agent-5744455dbd-b4d82\" (UID: \"d264612c-c1e6-4c14-b57d-11bf19bbd320\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:43.121005 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:43.120915 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:43.155750 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:43.155720 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" Jul 16 04:40:43.321866 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:43.321827 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:44.913697 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:44.913674 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6zfcn\" (UID: \"566deb82-6b60-4c93-aaeb-556a2e9bd270\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6zfcn" Jul 16 04:40:44.919545 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:44.916124 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/566deb82-6b60-4c93-aaeb-556a2e9bd270-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6zfcn\" (UID: \"566deb82-6b60-4c93-aaeb-556a2e9bd270\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6zfcn" Jul 16 04:40:44.966689 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:44.966653 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82"] Jul 16 04:40:44.969666 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:40:44.969636 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd264612c_c1e6_4c14_b57d_11bf19bbd320.slice/crio-7d5d317e82d7dc56a92efb45aa563f9f10f2c2357e2e13ebb9a892d9bfaa12c8 WatchSource:0}: Error finding container 7d5d317e82d7dc56a92efb45aa563f9f10f2c2357e2e13ebb9a892d9bfaa12c8: Status 404 returned error can't find the container with id 7d5d317e82d7dc56a92efb45aa563f9f10f2c2357e2e13ebb9a892d9bfaa12c8 Jul 16 04:40:45.117712 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:45.117636 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-6zfcn" Jul 16 04:40:45.258647 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:45.258611 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-6zfcn"] Jul 16 04:40:45.261693 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:40:45.261667 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod566deb82_6b60_4c93_aaeb_556a2e9bd270.slice/crio-78bfba6d4ed6180ae07dbeb871c94037e1071244cd68f42f7bb0e16b086ca066 WatchSource:0}: Error finding container 78bfba6d4ed6180ae07dbeb871c94037e1071244cd68f42f7bb0e16b086ca066: Status 404 returned error can't find the container with id 78bfba6d4ed6180ae07dbeb871c94037e1071244cd68f42f7bb0e16b086ca066 Jul 16 04:40:45.553708 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:45.553674 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" event={"ID":"d264612c-c1e6-4c14-b57d-11bf19bbd320","Type":"ContainerStarted","Data":"7d5d317e82d7dc56a92efb45aa563f9f10f2c2357e2e13ebb9a892d9bfaa12c8"} Jul 16 04:40:45.555513 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:45.555475 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w92t7" event={"ID":"794de458-b940-478f-a21b-9d2450761e3d","Type":"ContainerStarted","Data":"0ce5872b3afacea9c6a55db796a127cb098d60c0e6b52cfd9a6a6a70848a9e14"} Jul 16 04:40:45.556535 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:45.556498 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-6zfcn" event={"ID":"566deb82-6b60-4c93-aaeb-556a2e9bd270","Type":"ContainerStarted","Data":"78bfba6d4ed6180ae07dbeb871c94037e1071244cd68f42f7bb0e16b086ca066"} Jul 16 04:40:45.590627 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:45.590584 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-w92t7" podStartSLOduration=17.89236452 podStartE2EDuration="29.590570624s" totalImagesPullingTime="11.698206104s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:40:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:45.589401948 +0000 UTC m=+98.314870481" watchObservedRunningTime="2026-07-16 04:40:45.590570624 +0000 UTC m=+98.316039145" Jul 16 04:40:45.837292 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:45.837153 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:46.516837 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:46.516794 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:46.706688 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:46.706638 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:46.923070 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:46.922887 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:46.933615 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:46.933592 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:40:46.936394 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:46.936370 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6419d78a-657f-4e8c-8ea4-6f5a4f92c234-metrics-tls\") pod \"dns-default-hjvdt\" (UID: \"6419d78a-657f-4e8c-8ea4-6f5a4f92c234\") " pod="openshift-dns/dns-default-hjvdt" Jul 16 04:40:46.984048 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:46.983978 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pmfl6\"" Jul 16 04:40:46.990921 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:46.990892 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hjvdt" Jul 16 04:40:47.189593 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:47.189564 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:47.350298 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:47.350262 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hjvdt"] Jul 16 04:40:47.361138 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:40:47.361109 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6419d78a_657f_4e8c_8ea4_6f5a4f92c234.slice/crio-edc772e929175966343b316f854ac19f346d7f08c3229914fcbe0a9fc58f2269 WatchSource:0}: Error finding container edc772e929175966343b316f854ac19f346d7f08c3229914fcbe0a9fc58f2269: Status 404 returned error can't find the container with id edc772e929175966343b316f854ac19f346d7f08c3229914fcbe0a9fc58f2269 Jul 16 04:40:47.565059 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:47.565027 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hjvdt" event={"ID":"6419d78a-657f-4e8c-8ea4-6f5a4f92c234","Type":"ContainerStarted","Data":"edc772e929175966343b316f854ac19f346d7f08c3229914fcbe0a9fc58f2269"} Jul 16 04:40:47.566548 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:47.566521 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-6zfcn" event={"ID":"566deb82-6b60-4c93-aaeb-556a2e9bd270","Type":"ContainerStarted","Data":"cf28dafb33bf5669380e6f0fadac0ff278c563ebd3a8101215580d3cf77818d1"} Jul 16 04:40:47.599479 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:47.599427 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-6zfcn" podStartSLOduration=33.67319454 podStartE2EDuration="35.599406591s" totalImagesPullingTime="1.926212051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:47.598852785 +0000 UTC m=+100.324321344" watchObservedRunningTime="2026-07-16 04:40:47.599406591 +0000 UTC m=+100.324875114" Jul 16 04:40:47.908972 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:47.908880 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:48.087767 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:48.087719 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:48.258571 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:48.258534 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:48.946229 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:48.946180 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:49.079971 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:49.079936 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:49.575198 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:49.575170 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" event={"ID":"d264612c-c1e6-4c14-b57d-11bf19bbd320","Type":"ContainerStarted","Data":"72221a6a578c18148fc4c16b825839cb5c8703494844d6f1ccec4efe38bcfe48"} Jul 16 04:40:50.221068 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:50.221035 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:50.495375 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:50.495304 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:50.580039 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:50.580011 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" event={"ID":"d264612c-c1e6-4c14-b57d-11bf19bbd320","Type":"ContainerStarted","Data":"53364d086fd93ada5257813b17a6c81490e0d531f349c1420f957b9e9ae2c56a"} Jul 16 04:40:50.580171 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:50.580053 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" event={"ID":"d264612c-c1e6-4c14-b57d-11bf19bbd320","Type":"ContainerStarted","Data":"55073bd98adc4ecae0a8bb9c2af5f020ed844dab3896c5c3f5cd3f884a79c981"} Jul 16 04:40:50.609867 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:50.609824 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5744455dbd-b4d82" podStartSLOduration=4.18137591 podStartE2EDuration="8.609810648s" totalImagesPullingTime="4.428434738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:40:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:50.609097293 +0000 UTC m=+103.334565816" watchObservedRunningTime="2026-07-16 04:40:50.609810648 +0000 UTC m=+103.335279222" Jul 16 04:40:50.856015 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:50.855989 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:51.044187 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:51.044110 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:51.191491 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:51.191460 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:51.464077 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:51.464052 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bwl6r" Jul 16 04:40:51.584829 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:51.584798 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hjvdt" event={"ID":"6419d78a-657f-4e8c-8ea4-6f5a4f92c234","Type":"ContainerStarted","Data":"ddbd1b2c2048605815dddf1e65a2c878e01eda019dc080f6f15dfa6a82f04c6f"} Jul 16 04:40:51.584957 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:51.584841 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hjvdt" event={"ID":"6419d78a-657f-4e8c-8ea4-6f5a4f92c234","Type":"ContainerStarted","Data":"b5f6af1a5a1b1d2d60fdca253d51dafdd8e2dec88e2fa8afb324ac127d2832c9"} Jul 16 04:40:51.584957 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:51.584887 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hjvdt" Jul 16 04:40:51.618717 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:51.618674 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hjvdt" podStartSLOduration=65.272147228 podStartE2EDuration="1m8.618661356s" totalImagesPullingTime="3.346514128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:39:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:51.618652496 +0000 UTC m=+104.344121019" watchObservedRunningTime="2026-07-16 04:40:51.618661356 +0000 UTC m=+104.344129878" Jul 16 04:40:52.943598 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:52.943560 2581 patch_prober.go:28] interesting pod/image-registry-5ddb69fbb8-rqsd6 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 16 04:40:52.943944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:52.943605 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" podUID="c75845e3-f9c9-43e4-8ea1-e07a3671601a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 04:40:55.388040 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:55.388001 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:55.521738 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:55.521698 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:59.296568 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:59.296535 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:59.609627 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:59.609529 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:59.833774 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:59.833731 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:40:59.983091 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:40:59.983056 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:00.410711 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:00.410631 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:00.591436 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:00.591398 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:01.341001 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:01.340970 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:01.509094 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:01.509039 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:01.590373 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:01.590348 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hjvdt" Jul 16 04:41:01.970179 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:01.970149 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:02.117769 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:02.117739 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:02.942448 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:02.942421 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:41:03.353146 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:03.353107 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:05.008962 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:05.008919 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:05.149303 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:05.149274 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:06.270224 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:06.270187 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:06.728004 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:06.727974 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:07.115781 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.115714 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jkmvx"] Jul 16 04:41:07.117940 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.117924 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.134365 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.134340 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 04:41:07.134481 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.134399 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 04:41:07.137341 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.137325 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 04:41:07.137428 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.137328 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kdsbv\"" Jul 16 04:41:07.138110 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.138093 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 04:41:07.138206 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.138181 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 04:41:07.162376 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.162357 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 04:41:07.291285 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.291262 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-sys\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.291684 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.291290 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-node-exporter-textfile\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.291684 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.291309 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-node-exporter-tls\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.291684 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.291324 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.291684 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.291413 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-node-exporter-accelerators-collector-config\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.291684 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.291486 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-node-exporter-wtmp\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.291684 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.291526 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-root\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.291684 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.291550 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hdh7\" (UniqueName: \"kubernetes.io/projected/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-kube-api-access-9hdh7\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.291684 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.291568 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-metrics-client-ca\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.392222 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.392154 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-root\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.392222 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.392194 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hdh7\" (UniqueName: \"kubernetes.io/projected/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-kube-api-access-9hdh7\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.392222 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.392216 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-metrics-client-ca\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.392418 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.392264 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-root\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.392418 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.392269 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-sys\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.392418 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.392302 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-sys\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.392418 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.392309 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-node-exporter-textfile\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.392418 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.392361 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-node-exporter-tls\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.392418 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.392401 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.392740 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.392434 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-node-exporter-accelerators-collector-config\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.392740 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.392481 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-node-exporter-wtmp\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.392740 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.392639 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-node-exporter-textfile\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.392740 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.392653 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-node-exporter-wtmp\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.392976 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.392957 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-node-exporter-accelerators-collector-config\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.392976 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.392965 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-metrics-client-ca\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.394714 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.394689 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-node-exporter-tls\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.394714 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.394702 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.408060 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.408037 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:07.410655 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.410631 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hdh7\" (UniqueName: \"kubernetes.io/projected/b1c7ae8c-3aca-4b72-827d-672b3d14dea6-kube-api-access-9hdh7\") pod \"node-exporter-jkmvx\" (UID: \"b1c7ae8c-3aca-4b72-827d-672b3d14dea6\") " pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.427649 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.427631 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jkmvx" Jul 16 04:41:07.436521 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:41:07.436489 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1c7ae8c_3aca_4b72_827d_672b3d14dea6.slice/crio-6f2d6234eea5e9a14cdd32b8078e7b7854e1f951fe1fcaa58491435b32d91cfd WatchSource:0}: Error finding container 6f2d6234eea5e9a14cdd32b8078e7b7854e1f951fe1fcaa58491435b32d91cfd: Status 404 returned error can't find the container with id 6f2d6234eea5e9a14cdd32b8078e7b7854e1f951fe1fcaa58491435b32d91cfd Jul 16 04:41:07.641910 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.641880 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jkmvx" event={"ID":"b1c7ae8c-3aca-4b72-827d-672b3d14dea6","Type":"ContainerStarted","Data":"6f2d6234eea5e9a14cdd32b8078e7b7854e1f951fe1fcaa58491435b32d91cfd"} Jul 16 04:41:07.716125 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.716083 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:07.961527 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:07.961470 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" podUID="c75845e3-f9c9-43e4-8ea1-e07a3671601a" containerName="registry" containerID="cri-o://f8597d2214b1b9ca9727db5e9e090589ec7b59885abef9a533847ac2912c3163" gracePeriod=30 Jul 16 04:41:08.115986 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.115952 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:08.376675 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.376648 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:41:08.502287 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.502258 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c75845e3-f9c9-43e4-8ea1-e07a3671601a-ca-trust-extracted\") pod \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " Jul 16 04:41:08.502287 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.502292 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-bound-sa-token\") pod \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " Jul 16 04:41:08.502529 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.502322 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lr26k\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-kube-api-access-lr26k\") pod \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " Jul 16 04:41:08.502529 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.502353 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c75845e3-f9c9-43e4-8ea1-e07a3671601a-image-registry-private-configuration\") pod \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " Jul 16 04:41:08.502529 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.502404 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c75845e3-f9c9-43e4-8ea1-e07a3671601a-installation-pull-secrets\") pod \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " Jul 16 04:41:08.502687 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.502554 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c75845e3-f9c9-43e4-8ea1-e07a3671601a-trusted-ca\") pod \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " Jul 16 04:41:08.502687 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.502591 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls\") pod \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " Jul 16 04:41:08.502687 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.502622 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-certificates\") pod \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\" (UID: \"c75845e3-f9c9-43e4-8ea1-e07a3671601a\") " Jul 16 04:41:08.503028 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.502996 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c75845e3-f9c9-43e4-8ea1-e07a3671601a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c75845e3-f9c9-43e4-8ea1-e07a3671601a" (UID: "c75845e3-f9c9-43e4-8ea1-e07a3671601a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:41:08.503183 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.503159 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c75845e3-f9c9-43e4-8ea1-e07a3671601a" (UID: "c75845e3-f9c9-43e4-8ea1-e07a3671601a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:41:08.504697 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.504645 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c75845e3-f9c9-43e4-8ea1-e07a3671601a" (UID: "c75845e3-f9c9-43e4-8ea1-e07a3671601a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:41:08.504935 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.504896 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c75845e3-f9c9-43e4-8ea1-e07a3671601a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c75845e3-f9c9-43e4-8ea1-e07a3671601a" (UID: "c75845e3-f9c9-43e4-8ea1-e07a3671601a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:41:08.504935 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.504907 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-kube-api-access-lr26k" (OuterVolumeSpecName: "kube-api-access-lr26k") pod "c75845e3-f9c9-43e4-8ea1-e07a3671601a" (UID: "c75845e3-f9c9-43e4-8ea1-e07a3671601a"). InnerVolumeSpecName "kube-api-access-lr26k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:41:08.505209 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.505176 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c75845e3-f9c9-43e4-8ea1-e07a3671601a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c75845e3-f9c9-43e4-8ea1-e07a3671601a" (UID: "c75845e3-f9c9-43e4-8ea1-e07a3671601a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:41:08.505315 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.505259 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c75845e3-f9c9-43e4-8ea1-e07a3671601a" (UID: "c75845e3-f9c9-43e4-8ea1-e07a3671601a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:41:08.506481 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.506456 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:08.511954 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.511905 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c75845e3-f9c9-43e4-8ea1-e07a3671601a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c75845e3-f9c9-43e4-8ea1-e07a3671601a" (UID: "c75845e3-f9c9-43e4-8ea1-e07a3671601a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:41:08.604080 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.604058 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c75845e3-f9c9-43e4-8ea1-e07a3671601a-trusted-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:41:08.604080 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.604079 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-tls\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:41:08.604247 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.604088 2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c75845e3-f9c9-43e4-8ea1-e07a3671601a-registry-certificates\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:41:08.604247 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.604100 2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c75845e3-f9c9-43e4-8ea1-e07a3671601a-ca-trust-extracted\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:41:08.604247 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.604108 2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-bound-sa-token\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:41:08.604247 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.604118 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lr26k\" (UniqueName: \"kubernetes.io/projected/c75845e3-f9c9-43e4-8ea1-e07a3671601a-kube-api-access-lr26k\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:41:08.604247 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.604127 2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c75845e3-f9c9-43e4-8ea1-e07a3671601a-image-registry-private-configuration\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:41:08.604247 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.604136 2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c75845e3-f9c9-43e4-8ea1-e07a3671601a-installation-pull-secrets\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:41:08.646033 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.646006 2581 generic.go:356] "Generic (PLEG): container finished" podID="c75845e3-f9c9-43e4-8ea1-e07a3671601a" containerID="f8597d2214b1b9ca9727db5e9e090589ec7b59885abef9a533847ac2912c3163" exitCode=0 Jul 16 04:41:08.646171 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.646049 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" Jul 16 04:41:08.646171 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.646098 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" event={"ID":"c75845e3-f9c9-43e4-8ea1-e07a3671601a","Type":"ContainerDied","Data":"f8597d2214b1b9ca9727db5e9e090589ec7b59885abef9a533847ac2912c3163"} Jul 16 04:41:08.646171 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.646147 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5ddb69fbb8-rqsd6" event={"ID":"c75845e3-f9c9-43e4-8ea1-e07a3671601a","Type":"ContainerDied","Data":"53e4bad4d7aafa7cbfb5a61b271a46eb1947ec73787541d3fc1c43eb6c52bcd3"} Jul 16 04:41:08.646342 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.646175 2581 scope.go:117] "RemoveContainer" containerID="f8597d2214b1b9ca9727db5e9e090589ec7b59885abef9a533847ac2912c3163" Jul 16 04:41:08.647802 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.647777 2581 generic.go:356] "Generic (PLEG): container finished" podID="b1c7ae8c-3aca-4b72-827d-672b3d14dea6" containerID="3e4b75b99473d04e4065cc95fa09f2be0d7ebec509725a93d79b51cbfecfca59" exitCode=0 Jul 16 04:41:08.647876 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.647817 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jkmvx" event={"ID":"b1c7ae8c-3aca-4b72-827d-672b3d14dea6","Type":"ContainerDied","Data":"3e4b75b99473d04e4065cc95fa09f2be0d7ebec509725a93d79b51cbfecfca59"} Jul 16 04:41:08.654629 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.654612 2581 scope.go:117] "RemoveContainer" containerID="f8597d2214b1b9ca9727db5e9e090589ec7b59885abef9a533847ac2912c3163" Jul 16 04:41:08.655246 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:41:08.655207 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8597d2214b1b9ca9727db5e9e090589ec7b59885abef9a533847ac2912c3163\": container with ID starting with f8597d2214b1b9ca9727db5e9e090589ec7b59885abef9a533847ac2912c3163 not found: ID does not exist" containerID="f8597d2214b1b9ca9727db5e9e090589ec7b59885abef9a533847ac2912c3163" Jul 16 04:41:08.655357 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.655256 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8597d2214b1b9ca9727db5e9e090589ec7b59885abef9a533847ac2912c3163"} err="failed to get container status \"f8597d2214b1b9ca9727db5e9e090589ec7b59885abef9a533847ac2912c3163\": rpc error: code = NotFound desc = could not find container \"f8597d2214b1b9ca9727db5e9e090589ec7b59885abef9a533847ac2912c3163\": container with ID starting with f8597d2214b1b9ca9727db5e9e090589ec7b59885abef9a533847ac2912c3163 not found: ID does not exist" Jul 16 04:41:08.760498 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.760479 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5ddb69fbb8-rqsd6"] Jul 16 04:41:08.778147 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:08.778090 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5ddb69fbb8-rqsd6"] Jul 16 04:41:09.370649 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:09.370617 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:09.593716 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:09.593685 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:09.653547 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:09.653458 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jkmvx" event={"ID":"b1c7ae8c-3aca-4b72-827d-672b3d14dea6","Type":"ContainerStarted","Data":"d00f6f0d784a8108da8e558f095617905a53523aac9f5bc30bbd31b2ba30abf6"} Jul 16 04:41:09.653686 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:09.653573 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jkmvx" event={"ID":"b1c7ae8c-3aca-4b72-827d-672b3d14dea6","Type":"ContainerStarted","Data":"357e4c71dcb20ad0b4bbb024e9ceb461d61b695017f639a8581fad05fe77771f"} Jul 16 04:41:09.700258 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:09.700215 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jkmvx" podStartSLOduration=1.785085026 podStartE2EDuration="2.700199805s" totalImagesPullingTime="849.147213ms" totalInitContainerRuntime="65.967566ms" isStatefulPod=true podCreationTimestamp="2026-07-16 04:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:41:09.699748282 +0000 UTC m=+122.425216805" watchObservedRunningTime="2026-07-16 04:41:09.700199805 +0000 UTC m=+122.425668344" Jul 16 04:41:09.837336 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:09.837296 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c75845e3-f9c9-43e4-8ea1-e07a3671601a" path="/var/lib/kubelet/pods/c75845e3-f9c9-43e4-8ea1-e07a3671601a/volumes" Jul 16 04:41:10.246153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.246127 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-659f84fb8-fvjgx"] Jul 16 04:41:10.246413 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.246398 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c75845e3-f9c9-43e4-8ea1-e07a3671601a" containerName="registry" Jul 16 04:41:10.246476 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.246414 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c75845e3-f9c9-43e4-8ea1-e07a3671601a" containerName="registry" Jul 16 04:41:10.246476 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.246466 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c75845e3-f9c9-43e4-8ea1-e07a3671601a" containerName="registry" Jul 16 04:41:10.253777 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.253757 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.259496 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.259471 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 16 04:41:10.259632 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.259470 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 16 04:41:10.259632 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.259586 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 16 04:41:10.264557 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.264540 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 16 04:41:10.264798 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.264785 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1ot9itbk6bm25\"" Jul 16 04:41:10.271640 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.271620 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 16 04:41:10.271837 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.271819 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-d4l5j\"" Jul 16 04:41:10.300693 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.300667 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-659f84fb8-fvjgx"] Jul 16 04:41:10.418431 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.418400 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-secret-thanos-querier-tls\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.418612 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.418449 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.418612 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.418479 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.418612 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.418551 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.418612 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.418590 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-metrics-client-ca\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.418815 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.418621 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-secret-grpc-tls\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.418815 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.418668 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.418815 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.418696 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7mw5\" (UniqueName: \"kubernetes.io/projected/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-kube-api-access-f7mw5\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.519181 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.519094 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-secret-thanos-querier-tls\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.519444 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.519426 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.519494 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.519461 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.519725 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.519488 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.519725 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.519529 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-metrics-client-ca\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.519725 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.519559 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-secret-grpc-tls\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.519725 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.519602 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.519725 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.519627 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7mw5\" (UniqueName: \"kubernetes.io/projected/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-kube-api-access-f7mw5\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.520333 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.520292 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-metrics-client-ca\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.522386 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.522359 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.522956 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.522721 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-secret-thanos-querier-tls\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.522956 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.522894 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-secret-grpc-tls\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.522956 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.522914 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.523114 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.522987 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.523274 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.523253 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.529432 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.529409 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:10.552768 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.552746 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7mw5\" (UniqueName: \"kubernetes.io/projected/cc2ce8d2-bc2d-4380-a3e6-53532c708b62-kube-api-access-f7mw5\") pod \"thanos-querier-659f84fb8-fvjgx\" (UID: \"cc2ce8d2-bc2d-4380-a3e6-53532c708b62\") " pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.563120 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.563096 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:10.804374 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.804305 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-659f84fb8-fvjgx"] Jul 16 04:41:10.810042 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:41:10.810013 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc2ce8d2_bc2d_4380_a3e6_53532c708b62.slice/crio-d0212db26f128d4f7a39cca1c2a776e46c2c843fc252ec5d28ed62fae93fc5a6 WatchSource:0}: Error finding container d0212db26f128d4f7a39cca1c2a776e46c2c843fc252ec5d28ed62fae93fc5a6: Status 404 returned error can't find the container with id d0212db26f128d4f7a39cca1c2a776e46c2c843fc252ec5d28ed62fae93fc5a6 Jul 16 04:41:10.821589 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:10.821562 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:11.111901 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:11.111823 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:11.553065 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:11.553033 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:11.668331 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:11.668296 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" event={"ID":"cc2ce8d2-bc2d-4380-a3e6-53532c708b62","Type":"ContainerStarted","Data":"d0212db26f128d4f7a39cca1c2a776e46c2c843fc252ec5d28ed62fae93fc5a6"} Jul 16 04:41:11.822318 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:11.822132 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:13.419202 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:13.419104 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:13.564708 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:13.564675 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:13.677580 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:13.677485 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" event={"ID":"cc2ce8d2-bc2d-4380-a3e6-53532c708b62","Type":"ContainerStarted","Data":"37657deba3e9793704e5645e37049a32cb4fd717f49ea07852fad77c14ba0e88"} Jul 16 04:41:13.677580 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:13.677574 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" event={"ID":"cc2ce8d2-bc2d-4380-a3e6-53532c708b62","Type":"ContainerStarted","Data":"cdff78f61f8774314f287c911908bd792fea329541cf44ec6079e761ea14f446"} Jul 16 04:41:13.677752 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:13.677600 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" event={"ID":"cc2ce8d2-bc2d-4380-a3e6-53532c708b62","Type":"ContainerStarted","Data":"46e03b964df240e9677c3c3d4f1b0f75f3bfb6e72da3d013a363613f9cfd29e8"} Jul 16 04:41:14.035871 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:14.035821 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:15.431697 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:15.431617 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:15.593765 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:15.593723 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:15.689246 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:15.689167 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" event={"ID":"cc2ce8d2-bc2d-4380-a3e6-53532c708b62","Type":"ContainerStarted","Data":"b15d2597993f2d0b16f9c94ef681f828e9af8c7f74771c48f9b2cb87d09b4428"} Jul 16 04:41:15.689246 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:15.689243 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" event={"ID":"cc2ce8d2-bc2d-4380-a3e6-53532c708b62","Type":"ContainerStarted","Data":"95629bca61c2193f31eba62cfbfa57b79755760c865a90c79dd93eb613bb0bff"} Jul 16 04:41:15.689428 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:15.689285 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" event={"ID":"cc2ce8d2-bc2d-4380-a3e6-53532c708b62","Type":"ContainerStarted","Data":"e8553d6e3d4610730e6a53489d1fa55a25278e9275b5a21cb5858155f82b3c53"} Jul 16 04:41:15.689428 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:15.689332 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:15.734760 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:15.734713 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" podStartSLOduration=2.151114257 podStartE2EDuration="5.734698201s" totalImagesPullingTime="3.583583944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:41:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:41:15.732907431 +0000 UTC m=+128.458375964" watchObservedRunningTime="2026-07-16 04:41:15.734698201 +0000 UTC m=+128.460166723" Jul 16 04:41:17.678338 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:17.678300 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs\") pod \"network-metrics-daemon-7g5n5\" (UID: \"2f51d802-9618-4e88-8fc8-78b2a3788356\") " pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:41:17.680679 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:17.680655 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f51d802-9618-4e88-8fc8-78b2a3788356-metrics-certs\") pod \"network-metrics-daemon-7g5n5\" (UID: \"2f51d802-9618-4e88-8fc8-78b2a3788356\") " pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:41:17.853372 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:17.853348 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6n555\"" Jul 16 04:41:17.855437 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:17.855415 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7g5n5" Jul 16 04:41:18.155101 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:18.155071 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7g5n5"] Jul 16 04:41:18.158283 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:41:18.158255 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f51d802_9618_4e88_8fc8_78b2a3788356.slice/crio-4963325d5f627f17960535e4909385435447cc1bd610e5cafed31ee9de958737 WatchSource:0}: Error finding container 4963325d5f627f17960535e4909385435447cc1bd610e5cafed31ee9de958737: Status 404 returned error can't find the container with id 4963325d5f627f17960535e4909385435447cc1bd610e5cafed31ee9de958737 Jul 16 04:41:18.702153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:18.702112 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7g5n5" event={"ID":"2f51d802-9618-4e88-8fc8-78b2a3788356","Type":"ContainerStarted","Data":"4963325d5f627f17960535e4909385435447cc1bd610e5cafed31ee9de958737"} Jul 16 04:41:19.711096 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:19.711061 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7g5n5" event={"ID":"2f51d802-9618-4e88-8fc8-78b2a3788356","Type":"ContainerStarted","Data":"8738e2d63d323cf3a1b1f3ba68b818df7e52d764a8d8e17a060628edfcff4877"} Jul 16 04:41:20.716203 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:20.716173 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7g5n5" event={"ID":"2f51d802-9618-4e88-8fc8-78b2a3788356","Type":"ContainerStarted","Data":"9a4201dc44d40dc301a657b5a211b2ff2f33e65fb933c63b2762392c5816e4be"} Jul 16 04:41:21.000338 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:21.000264 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:21.240229 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:21.240193 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:21.700000 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:21.699974 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-659f84fb8-fvjgx" Jul 16 04:41:21.903184 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:21.903136 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7g5n5" podStartSLOduration=132.548958542 podStartE2EDuration="2m13.903121612s" totalImagesPullingTime="1.35416307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:41:20.752358344 +0000 UTC m=+133.477826867" watchObservedRunningTime="2026-07-16 04:41:21.903121612 +0000 UTC m=+134.628590181" Jul 16 04:41:24.212247 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:24.212214 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:24.493489 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:24.493412 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:24.988294 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:24.988256 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:25.175367 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:25.175339 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:25.377555 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:25.377456 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:25.544253 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:25.544218 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:26.737936 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:26.737906 2581 generic.go:356] "Generic (PLEG): container finished" podID="aaa2dfa0-68ee-427d-86dc-bb788a81aec5" containerID="f3a5be60417895213c242b60324d75f6bc15c74b7b45d70a328d1fb181efeede" exitCode=0 Jul 16 04:41:26.738321 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:26.737987 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" event={"ID":"aaa2dfa0-68ee-427d-86dc-bb788a81aec5","Type":"ContainerDied","Data":"f3a5be60417895213c242b60324d75f6bc15c74b7b45d70a328d1fb181efeede"} Jul 16 04:41:26.738321 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:26.738317 2581 scope.go:117] "RemoveContainer" containerID="f3a5be60417895213c242b60324d75f6bc15c74b7b45d70a328d1fb181efeede" Jul 16 04:41:27.682470 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:27.682440 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kdmwr_2f60f16a-e6e4-4289-b19b-18c9c3826d21/serve-healthcheck-canary/0.log" Jul 16 04:41:27.743395 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:27.743361 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-cgw2x" event={"ID":"aaa2dfa0-68ee-427d-86dc-bb788a81aec5","Type":"ContainerStarted","Data":"dceac7e7cc8f57d57eddd57879af51430c7fbee2b60addc7447eadc4f39749ab"} Jul 16 04:41:28.462684 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:28.462658 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:28.567718 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:28.567687 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:32.265120 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:32.265078 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:32.543605 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:32.543493 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:32.738861 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:32.738830 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:40.435355 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:40.435283 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:40.595707 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:40.595553 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:41:47.819093 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:47.819059 2581 generic.go:356] "Generic (PLEG): container finished" podID="fb7574c6-5d40-40ed-b909-0abde38d01ba" containerID="0f15e6a83987c86e1c3d92c4bb9ec2861c61692dacd960bcc1074a2a0d01a8cf" exitCode=0 Jul 16 04:41:47.819612 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:47.819139 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8" event={"ID":"fb7574c6-5d40-40ed-b909-0abde38d01ba","Type":"ContainerDied","Data":"0f15e6a83987c86e1c3d92c4bb9ec2861c61692dacd960bcc1074a2a0d01a8cf"} Jul 16 04:41:47.819612 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:47.819557 2581 scope.go:117] "RemoveContainer" containerID="0f15e6a83987c86e1c3d92c4bb9ec2861c61692dacd960bcc1074a2a0d01a8cf" Jul 16 04:41:48.823891 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:48.823857 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-6vqq8" event={"ID":"fb7574c6-5d40-40ed-b909-0abde38d01ba","Type":"ContainerStarted","Data":"6af4b76b9c1dae6a32192302d15168ed6336007cc8e56e605b692ce0424ca7f4"} Jul 16 04:41:51.888431 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:41:51.888391 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:42:23.060047 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:42:23.060008 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:42:23.820395 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:42:23.820342 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:42:23.946566 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:42:23.946534 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:42:32.598994 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:42:32.598952 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:42:35.791640 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:42:35.791598 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:42:35.908254 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:42:35.908222 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:42:40.484102 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:42:40.484063 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:42:40.595639 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:42:40.595608 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:42:42.156149 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:42:42.156110 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:42:52.785144 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:42:52.785102 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:42:52.885250 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:42:52.885216 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:42:57.758028 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:42:57.757987 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:42:57.823965 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:42:57.823925 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:43:23.505470 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:43:23.505437 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:43:23.607562 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:43:23.607528 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:43:50.942565 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:43:50.942529 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:43:51.041637 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:43:51.041606 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:43:54.945688 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:43:54.945656 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:43:55.039478 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:43:55.039441 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:43:56.244723 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:43:56.244683 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:43:56.452236 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:43:56.452172 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:43:56.808777 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:43:56.808736 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:43:56.959394 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:43:56.959367 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:43:59.385997 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:43:59.385959 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:43:59.557161 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:43:59.557131 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:00.918057 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:00.918023 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:01.017559 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:01.017526 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:06.600647 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:06.600609 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:07.706149 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:07.706116 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrbbv_95a3a732-50cf-47f5-9629-0b3d241d017a/ovn-acl-logging/0.log" Jul 16 04:44:07.713201 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:07.713177 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrbbv_95a3a732-50cf-47f5-9629-0b3d241d017a/ovn-acl-logging/0.log" Jul 16 04:44:07.714805 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:07.714780 2581 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 04:44:07.925367 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:07.925175 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:08.099128 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:08.099099 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:09.179601 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:09.179555 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6dhq2"] Jul 16 04:44:09.182604 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:09.182585 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6dhq2" Jul 16 04:44:09.185349 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:09.185326 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 16 04:44:09.186150 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:09.186134 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 16 04:44:09.186201 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:09.186134 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-48jjb\"" Jul 16 04:44:09.194460 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:09.194437 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6dhq2"] Jul 16 04:44:09.321584 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:09.321552 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmsfk\" (UniqueName: \"kubernetes.io/projected/2516dbd5-08da-4e57-b9f5-c3bab3f1c50e-kube-api-access-zmsfk\") pod \"openshift-lws-operator-fd8ccff4c-6dhq2\" (UID: \"2516dbd5-08da-4e57-b9f5-c3bab3f1c50e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6dhq2" Jul 16 04:44:09.321584 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:09.321586 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2516dbd5-08da-4e57-b9f5-c3bab3f1c50e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-6dhq2\" (UID: \"2516dbd5-08da-4e57-b9f5-c3bab3f1c50e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6dhq2" Jul 16 04:44:09.422002 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:09.421965 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmsfk\" (UniqueName: \"kubernetes.io/projected/2516dbd5-08da-4e57-b9f5-c3bab3f1c50e-kube-api-access-zmsfk\") pod \"openshift-lws-operator-fd8ccff4c-6dhq2\" (UID: \"2516dbd5-08da-4e57-b9f5-c3bab3f1c50e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6dhq2" Jul 16 04:44:09.422002 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:09.422007 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2516dbd5-08da-4e57-b9f5-c3bab3f1c50e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-6dhq2\" (UID: \"2516dbd5-08da-4e57-b9f5-c3bab3f1c50e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6dhq2" Jul 16 04:44:09.422340 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:09.422324 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2516dbd5-08da-4e57-b9f5-c3bab3f1c50e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-6dhq2\" (UID: \"2516dbd5-08da-4e57-b9f5-c3bab3f1c50e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6dhq2" Jul 16 04:44:09.431348 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:09.431271 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmsfk\" (UniqueName: \"kubernetes.io/projected/2516dbd5-08da-4e57-b9f5-c3bab3f1c50e-kube-api-access-zmsfk\") pod \"openshift-lws-operator-fd8ccff4c-6dhq2\" (UID: \"2516dbd5-08da-4e57-b9f5-c3bab3f1c50e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6dhq2" Jul 16 04:44:09.491177 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:09.491140 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6dhq2" Jul 16 04:44:09.626477 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:09.626456 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6dhq2"] Jul 16 04:44:09.628748 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:44:09.628709 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2516dbd5_08da_4e57_b9f5_c3bab3f1c50e.slice/crio-13390d763080794e7bf701a54e282b4ddd23e2584edfc0afc546435697dd7666 WatchSource:0}: Error finding container 13390d763080794e7bf701a54e282b4ddd23e2584edfc0afc546435697dd7666: Status 404 returned error can't find the container with id 13390d763080794e7bf701a54e282b4ddd23e2584edfc0afc546435697dd7666 Jul 16 04:44:09.630260 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:09.630243 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 04:44:10.314535 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:10.314454 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6dhq2" event={"ID":"2516dbd5-08da-4e57-b9f5-c3bab3f1c50e","Type":"ContainerStarted","Data":"13390d763080794e7bf701a54e282b4ddd23e2584edfc0afc546435697dd7666"} Jul 16 04:44:12.323034 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:12.322996 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6dhq2" event={"ID":"2516dbd5-08da-4e57-b9f5-c3bab3f1c50e","Type":"ContainerStarted","Data":"74a64ea414ee8400dcb4fc2effd00d233710da9821b19f28f40f7f775a93e433"} Jul 16 04:44:13.277086 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:13.277047 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:13.386934 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:13.386897 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:13.641720 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:13.641633 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:13.738351 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:13.738314 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:15.403898 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:15.403865 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:29.984089 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:29.984046 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:30.094776 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:30.094732 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:30.214015 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:30.213970 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:30.785409 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:30.785379 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:31.178658 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.178567 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:31.292279 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.292246 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:31.342858 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.342811 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6dhq2" podStartSLOduration=20.085650248 podStartE2EDuration="22.342795134s" totalImagesPullingTime="2.257144886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:44:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:44:12.345149992 +0000 UTC m=+305.070618540" watchObservedRunningTime="2026-07-16 04:44:31.342795134 +0000 UTC m=+324.068263656" Jul 16 04:44:31.343247 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.343232 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f"] Jul 16 04:44:31.346516 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.346485 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:31.349127 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.349104 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 16 04:44:31.349297 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.349275 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 16 04:44:31.349459 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.349442 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 16 04:44:31.349531 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.349519 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 16 04:44:31.349573 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.349551 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-x5n9f\"" Jul 16 04:44:31.367092 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.367068 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcg2p\" (UniqueName: \"kubernetes.io/projected/48e126e9-56a9-43eb-b98b-b6494445cbeb-kube-api-access-tcg2p\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-l262f\" (UID: \"48e126e9-56a9-43eb-b98b-b6494445cbeb\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:31.367184 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.367104 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48e126e9-56a9-43eb-b98b-b6494445cbeb-tmp\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-l262f\" (UID: \"48e126e9-56a9-43eb-b98b-b6494445cbeb\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:31.367184 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.367125 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48e126e9-56a9-43eb-b98b-b6494445cbeb-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-l262f\" (UID: \"48e126e9-56a9-43eb-b98b-b6494445cbeb\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:31.367254 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.367186 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/48e126e9-56a9-43eb-b98b-b6494445cbeb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-l262f\" (UID: \"48e126e9-56a9-43eb-b98b-b6494445cbeb\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:31.367254 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.367233 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/48e126e9-56a9-43eb-b98b-b6494445cbeb-manifests\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-l262f\" (UID: \"48e126e9-56a9-43eb-b98b-b6494445cbeb\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:31.372278 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.372253 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f"] Jul 16 04:44:31.468118 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.468083 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/48e126e9-56a9-43eb-b98b-b6494445cbeb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-l262f\" (UID: \"48e126e9-56a9-43eb-b98b-b6494445cbeb\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:31.468284 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.468141 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/48e126e9-56a9-43eb-b98b-b6494445cbeb-manifests\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-l262f\" (UID: \"48e126e9-56a9-43eb-b98b-b6494445cbeb\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:31.468284 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.468169 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcg2p\" (UniqueName: \"kubernetes.io/projected/48e126e9-56a9-43eb-b98b-b6494445cbeb-kube-api-access-tcg2p\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-l262f\" (UID: \"48e126e9-56a9-43eb-b98b-b6494445cbeb\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:31.468284 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.468192 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48e126e9-56a9-43eb-b98b-b6494445cbeb-tmp\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-l262f\" (UID: \"48e126e9-56a9-43eb-b98b-b6494445cbeb\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:31.468284 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.468221 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48e126e9-56a9-43eb-b98b-b6494445cbeb-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-l262f\" (UID: \"48e126e9-56a9-43eb-b98b-b6494445cbeb\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:31.468643 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.468621 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/48e126e9-56a9-43eb-b98b-b6494445cbeb-manifests\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-l262f\" (UID: \"48e126e9-56a9-43eb-b98b-b6494445cbeb\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:31.470412 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.470393 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48e126e9-56a9-43eb-b98b-b6494445cbeb-tmp\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-l262f\" (UID: \"48e126e9-56a9-43eb-b98b-b6494445cbeb\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:31.470643 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.470625 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48e126e9-56a9-43eb-b98b-b6494445cbeb-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-l262f\" (UID: \"48e126e9-56a9-43eb-b98b-b6494445cbeb\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:31.470698 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.470647 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/48e126e9-56a9-43eb-b98b-b6494445cbeb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-l262f\" (UID: \"48e126e9-56a9-43eb-b98b-b6494445cbeb\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:31.490934 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.490906 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcg2p\" (UniqueName: \"kubernetes.io/projected/48e126e9-56a9-43eb-b98b-b6494445cbeb-kube-api-access-tcg2p\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-l262f\" (UID: \"48e126e9-56a9-43eb-b98b-b6494445cbeb\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:31.571095 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.571048 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:31.656951 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.656916 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:31.669319 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.669290 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:31.813744 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:44:31.813704 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48e126e9_56a9_43eb_b98b_b6494445cbeb.slice/crio-5a8b66555247a752cb55701f6a4906994cd3ed1973153077b5c8a5aed6ccfa21 WatchSource:0}: Error finding container 5a8b66555247a752cb55701f6a4906994cd3ed1973153077b5c8a5aed6ccfa21: Status 404 returned error can't find the container with id 5a8b66555247a752cb55701f6a4906994cd3ed1973153077b5c8a5aed6ccfa21 Jul 16 04:44:31.814101 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:31.814081 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f"] Jul 16 04:44:32.396197 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:32.396160 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" event={"ID":"48e126e9-56a9-43eb-b98b-b6494445cbeb","Type":"ContainerStarted","Data":"5a8b66555247a752cb55701f6a4906994cd3ed1973153077b5c8a5aed6ccfa21"} Jul 16 04:44:33.162386 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:33.162352 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:35.410114 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:35.410078 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" event={"ID":"48e126e9-56a9-43eb-b98b-b6494445cbeb","Type":"ContainerStarted","Data":"51cdb310af859531cfb80b3da49e2c6dd9b585b2bea9cbd58967878988553911"} Jul 16 04:44:36.415537 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:36.415489 2581 generic.go:356] "Generic (PLEG): container finished" podID="48e126e9-56a9-43eb-b98b-b6494445cbeb" containerID="51cdb310af859531cfb80b3da49e2c6dd9b585b2bea9cbd58967878988553911" exitCode=0 Jul 16 04:44:36.415537 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:36.415530 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" event={"ID":"48e126e9-56a9-43eb-b98b-b6494445cbeb","Type":"ContainerDied","Data":"51cdb310af859531cfb80b3da49e2c6dd9b585b2bea9cbd58967878988553911"} Jul 16 04:44:37.421128 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:37.421088 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" event={"ID":"48e126e9-56a9-43eb-b98b-b6494445cbeb","Type":"ContainerStarted","Data":"2384aa4ade438f7f4e85a761b41213ff5ac2b4fed8a71ab6c45e73e955694c44"} Jul 16 04:44:37.421549 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:37.421194 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:37.452406 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:37.452348 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" podStartSLOduration=2.204689229 podStartE2EDuration="6.452334412s" totalImagesPullingTime="3.242247544s" totalInitContainerRuntime="1.005397639s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:44:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:44:37.450952596 +0000 UTC m=+330.176421118" watchObservedRunningTime="2026-07-16 04:44:37.452334412 +0000 UTC m=+330.177802933" Jul 16 04:44:39.571597 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:39.571556 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:39.677308 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:39.677269 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:48.428038 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:48.428007 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-l262f" Jul 16 04:44:50.709419 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:50.709381 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:51.010353 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.010254 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:51.420271 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.420198 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:51.533780 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.533741 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-68764b5749-fddrb"] Jul 16 04:44:51.546700 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.546668 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68764b5749-fddrb" Jul 16 04:44:51.551526 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.551494 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 16 04:44:51.557733 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.557712 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-84b2h\"" Jul 16 04:44:51.557910 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.557895 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 04:44:51.558007 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.557989 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 16 04:44:51.558740 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.558724 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 04:44:51.567129 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.567099 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68764b5749-fddrb"] Jul 16 04:44:51.601686 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.601650 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:51.623643 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.623620 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9351e506-d7ac-4be8-a5dc-84f1d25515ef-tls-certs\") pod \"kube-auth-proxy-68764b5749-fddrb\" (UID: \"9351e506-d7ac-4be8-a5dc-84f1d25515ef\") " pod="openshift-ingress/kube-auth-proxy-68764b5749-fddrb" Jul 16 04:44:51.623727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.623670 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9351e506-d7ac-4be8-a5dc-84f1d25515ef-tmp\") pod \"kube-auth-proxy-68764b5749-fddrb\" (UID: \"9351e506-d7ac-4be8-a5dc-84f1d25515ef\") " pod="openshift-ingress/kube-auth-proxy-68764b5749-fddrb" Jul 16 04:44:51.623727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.623692 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x59pp\" (UniqueName: \"kubernetes.io/projected/9351e506-d7ac-4be8-a5dc-84f1d25515ef-kube-api-access-x59pp\") pod \"kube-auth-proxy-68764b5749-fddrb\" (UID: \"9351e506-d7ac-4be8-a5dc-84f1d25515ef\") " pod="openshift-ingress/kube-auth-proxy-68764b5749-fddrb" Jul 16 04:44:51.724655 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.724629 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9351e506-d7ac-4be8-a5dc-84f1d25515ef-tls-certs\") pod \"kube-auth-proxy-68764b5749-fddrb\" (UID: \"9351e506-d7ac-4be8-a5dc-84f1d25515ef\") " pod="openshift-ingress/kube-auth-proxy-68764b5749-fddrb" Jul 16 04:44:51.725078 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.724683 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9351e506-d7ac-4be8-a5dc-84f1d25515ef-tmp\") pod \"kube-auth-proxy-68764b5749-fddrb\" (UID: \"9351e506-d7ac-4be8-a5dc-84f1d25515ef\") " pod="openshift-ingress/kube-auth-proxy-68764b5749-fddrb" Jul 16 04:44:51.725078 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.724716 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x59pp\" (UniqueName: \"kubernetes.io/projected/9351e506-d7ac-4be8-a5dc-84f1d25515ef-kube-api-access-x59pp\") pod \"kube-auth-proxy-68764b5749-fddrb\" (UID: \"9351e506-d7ac-4be8-a5dc-84f1d25515ef\") " pod="openshift-ingress/kube-auth-proxy-68764b5749-fddrb" Jul 16 04:44:51.726933 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.726902 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9351e506-d7ac-4be8-a5dc-84f1d25515ef-tmp\") pod \"kube-auth-proxy-68764b5749-fddrb\" (UID: \"9351e506-d7ac-4be8-a5dc-84f1d25515ef\") " pod="openshift-ingress/kube-auth-proxy-68764b5749-fddrb" Jul 16 04:44:51.727131 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.727111 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9351e506-d7ac-4be8-a5dc-84f1d25515ef-tls-certs\") pod \"kube-auth-proxy-68764b5749-fddrb\" (UID: \"9351e506-d7ac-4be8-a5dc-84f1d25515ef\") " pod="openshift-ingress/kube-auth-proxy-68764b5749-fddrb" Jul 16 04:44:51.750352 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.750321 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x59pp\" (UniqueName: \"kubernetes.io/projected/9351e506-d7ac-4be8-a5dc-84f1d25515ef-kube-api-access-x59pp\") pod \"kube-auth-proxy-68764b5749-fddrb\" (UID: \"9351e506-d7ac-4be8-a5dc-84f1d25515ef\") " pod="openshift-ingress/kube-auth-proxy-68764b5749-fddrb" Jul 16 04:44:51.856788 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.856758 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68764b5749-fddrb" Jul 16 04:44:51.873590 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.873565 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2q482"] Jul 16 04:44:51.876122 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.876097 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2q482" Jul 16 04:44:51.878964 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.878937 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 16 04:44:51.879063 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.878970 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-dgh5g\"" Jul 16 04:44:51.907194 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.907164 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2q482"] Jul 16 04:44:51.926801 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.926774 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5-cert\") pod \"odh-model-controller-7998c67dc4-2q482\" (UID: \"4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5\") " pod="opendatahub/odh-model-controller-7998c67dc4-2q482" Jul 16 04:44:51.926945 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:51.926849 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htkj7\" (UniqueName: \"kubernetes.io/projected/4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5-kube-api-access-htkj7\") pod \"odh-model-controller-7998c67dc4-2q482\" (UID: \"4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5\") " pod="opendatahub/odh-model-controller-7998c67dc4-2q482" Jul 16 04:44:52.025599 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:52.025515 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:52.027476 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:52.027448 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htkj7\" (UniqueName: \"kubernetes.io/projected/4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5-kube-api-access-htkj7\") pod \"odh-model-controller-7998c67dc4-2q482\" (UID: \"4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5\") " pod="opendatahub/odh-model-controller-7998c67dc4-2q482" Jul 16 04:44:52.027612 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:52.027557 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5-cert\") pod \"odh-model-controller-7998c67dc4-2q482\" (UID: \"4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5\") " pod="opendatahub/odh-model-controller-7998c67dc4-2q482" Jul 16 04:44:52.027707 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:44:52.027692 2581 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 16 04:44:52.027781 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:44:52.027770 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5-cert podName:4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5 nodeName:}" failed. No retries permitted until 2026-07-16 04:44:52.527749271 +0000 UTC m=+345.253217774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5-cert") pod "odh-model-controller-7998c67dc4-2q482" (UID: "4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5") : secret "odh-model-controller-webhook-cert" not found Jul 16 04:44:52.074277 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:52.074248 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68764b5749-fddrb"] Jul 16 04:44:52.075305 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:52.074819 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htkj7\" (UniqueName: \"kubernetes.io/projected/4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5-kube-api-access-htkj7\") pod \"odh-model-controller-7998c67dc4-2q482\" (UID: \"4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5\") " pod="opendatahub/odh-model-controller-7998c67dc4-2q482" Jul 16 04:44:52.080931 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:44:52.080903 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9351e506_d7ac_4be8_a5dc_84f1d25515ef.slice/crio-0004d14c95950dcbcb6b14d9f9bf3e333be2d5b5d527d9eb9e8e5805cef23a76 WatchSource:0}: Error finding container 0004d14c95950dcbcb6b14d9f9bf3e333be2d5b5d527d9eb9e8e5805cef23a76: Status 404 returned error can't find the container with id 0004d14c95950dcbcb6b14d9f9bf3e333be2d5b5d527d9eb9e8e5805cef23a76 Jul 16 04:44:52.242524 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:52.242464 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:52.421808 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:52.421730 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:52.474705 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:52.474666 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68764b5749-fddrb" event={"ID":"9351e506-d7ac-4be8-a5dc-84f1d25515ef","Type":"ContainerStarted","Data":"0004d14c95950dcbcb6b14d9f9bf3e333be2d5b5d527d9eb9e8e5805cef23a76"} Jul 16 04:44:52.531377 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:52.531346 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5-cert\") pod \"odh-model-controller-7998c67dc4-2q482\" (UID: \"4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5\") " pod="opendatahub/odh-model-controller-7998c67dc4-2q482" Jul 16 04:44:52.533666 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:52.533649 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5-cert\") pod \"odh-model-controller-7998c67dc4-2q482\" (UID: \"4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5\") " pod="opendatahub/odh-model-controller-7998c67dc4-2q482" Jul 16 04:44:52.787522 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:52.787468 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2q482" Jul 16 04:44:52.953784 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:52.953744 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2q482"] Jul 16 04:44:52.959792 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:44:52.959758 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ee5c1cd_0c5a_4cf7_b61f_e87d3f687ab5.slice/crio-33de5b41a952da04c15d3a5fcad724fcfa2bf028adb7cdc637937e764641b463 WatchSource:0}: Error finding container 33de5b41a952da04c15d3a5fcad724fcfa2bf028adb7cdc637937e764641b463: Status 404 returned error can't find the container with id 33de5b41a952da04c15d3a5fcad724fcfa2bf028adb7cdc637937e764641b463 Jul 16 04:44:53.162723 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:53.162605 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xgwlw"] Jul 16 04:44:53.165380 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:53.165360 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-xgwlw" Jul 16 04:44:53.170172 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:53.170140 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 16 04:44:53.182516 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:53.182477 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:53.185318 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:53.185292 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xgwlw"] Jul 16 04:44:53.236597 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:53.236570 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39f2d9d3-4b27-4b65-9725-7b30babbe5e5-tls-certs\") pod \"model-serving-api-54d89bd5fc-xgwlw\" (UID: \"39f2d9d3-4b27-4b65-9725-7b30babbe5e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-xgwlw" Jul 16 04:44:53.236732 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:53.236699 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ffpk\" (UniqueName: \"kubernetes.io/projected/39f2d9d3-4b27-4b65-9725-7b30babbe5e5-kube-api-access-7ffpk\") pod \"model-serving-api-54d89bd5fc-xgwlw\" (UID: \"39f2d9d3-4b27-4b65-9725-7b30babbe5e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-xgwlw" Jul 16 04:44:53.337471 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:53.337433 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ffpk\" (UniqueName: \"kubernetes.io/projected/39f2d9d3-4b27-4b65-9725-7b30babbe5e5-kube-api-access-7ffpk\") pod \"model-serving-api-54d89bd5fc-xgwlw\" (UID: \"39f2d9d3-4b27-4b65-9725-7b30babbe5e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-xgwlw" Jul 16 04:44:53.337669 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:53.337481 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39f2d9d3-4b27-4b65-9725-7b30babbe5e5-tls-certs\") pod \"model-serving-api-54d89bd5fc-xgwlw\" (UID: \"39f2d9d3-4b27-4b65-9725-7b30babbe5e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-xgwlw" Jul 16 04:44:53.337669 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:44:53.337610 2581 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 16 04:44:53.337778 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:44:53.337677 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39f2d9d3-4b27-4b65-9725-7b30babbe5e5-tls-certs podName:39f2d9d3-4b27-4b65-9725-7b30babbe5e5 nodeName:}" failed. No retries permitted until 2026-07-16 04:44:53.83766076 +0000 UTC m=+346.563129260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/39f2d9d3-4b27-4b65-9725-7b30babbe5e5-tls-certs") pod "model-serving-api-54d89bd5fc-xgwlw" (UID: "39f2d9d3-4b27-4b65-9725-7b30babbe5e5") : secret "model-serving-api-tls" not found Jul 16 04:44:53.362776 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:53.362714 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ffpk\" (UniqueName: \"kubernetes.io/projected/39f2d9d3-4b27-4b65-9725-7b30babbe5e5-kube-api-access-7ffpk\") pod \"model-serving-api-54d89bd5fc-xgwlw\" (UID: \"39f2d9d3-4b27-4b65-9725-7b30babbe5e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-xgwlw" Jul 16 04:44:53.480946 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:53.480907 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2q482" event={"ID":"4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5","Type":"ContainerStarted","Data":"33de5b41a952da04c15d3a5fcad724fcfa2bf028adb7cdc637937e764641b463"} Jul 16 04:44:53.843804 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:53.843204 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39f2d9d3-4b27-4b65-9725-7b30babbe5e5-tls-certs\") pod \"model-serving-api-54d89bd5fc-xgwlw\" (UID: \"39f2d9d3-4b27-4b65-9725-7b30babbe5e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-xgwlw" Jul 16 04:44:53.843804 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:44:53.843435 2581 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 16 04:44:53.843804 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:44:53.843516 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39f2d9d3-4b27-4b65-9725-7b30babbe5e5-tls-certs podName:39f2d9d3-4b27-4b65-9725-7b30babbe5e5 nodeName:}" failed. No retries permitted until 2026-07-16 04:44:54.843480163 +0000 UTC m=+347.568948666 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/39f2d9d3-4b27-4b65-9725-7b30babbe5e5-tls-certs") pod "model-serving-api-54d89bd5fc-xgwlw" (UID: "39f2d9d3-4b27-4b65-9725-7b30babbe5e5") : secret "model-serving-api-tls" not found Jul 16 04:44:54.016153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.015927 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j"] Jul 16 04:44:54.017960 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.017928 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:54.018483 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.018449 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.021108 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.021087 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 16 04:44:54.021394 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.021377 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-8w6rd\"" Jul 16 04:44:54.021594 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.021574 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 16 04:44:54.021594 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.021579 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 16 04:44:54.021737 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.021638 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 04:44:54.038596 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.038480 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j"] Jul 16 04:44:54.114568 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.114403 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:54.150707 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.150671 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/56afff30-8548-4dbc-afee-61ee544dfe22-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.150876 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.150726 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsp8f\" (UniqueName: \"kubernetes.io/projected/56afff30-8548-4dbc-afee-61ee544dfe22-kube-api-access-nsp8f\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.150876 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.150802 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56afff30-8548-4dbc-afee-61ee544dfe22-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.150876 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.150840 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/56afff30-8548-4dbc-afee-61ee544dfe22-cacerts\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.151032 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.150888 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/56afff30-8548-4dbc-afee-61ee544dfe22-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.151032 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.150923 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/56afff30-8548-4dbc-afee-61ee544dfe22-local-certs\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.151032 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.150951 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56afff30-8548-4dbc-afee-61ee544dfe22-istio-token\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.252085 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.252009 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56afff30-8548-4dbc-afee-61ee544dfe22-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.252085 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.252059 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/56afff30-8548-4dbc-afee-61ee544dfe22-cacerts\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.252375 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.252115 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/56afff30-8548-4dbc-afee-61ee544dfe22-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.252375 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.252150 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/56afff30-8548-4dbc-afee-61ee544dfe22-local-certs\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.252375 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.252176 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56afff30-8548-4dbc-afee-61ee544dfe22-istio-token\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.253373 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.253057 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/56afff30-8548-4dbc-afee-61ee544dfe22-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.253373 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.253119 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsp8f\" (UniqueName: \"kubernetes.io/projected/56afff30-8548-4dbc-afee-61ee544dfe22-kube-api-access-nsp8f\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.253941 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.253840 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/56afff30-8548-4dbc-afee-61ee544dfe22-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.256528 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.255786 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:54.256939 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.256871 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/56afff30-8548-4dbc-afee-61ee544dfe22-local-certs\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.257672 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.257632 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/56afff30-8548-4dbc-afee-61ee544dfe22-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.264451 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.264365 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56afff30-8548-4dbc-afee-61ee544dfe22-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.265815 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.265755 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsp8f\" (UniqueName: \"kubernetes.io/projected/56afff30-8548-4dbc-afee-61ee544dfe22-kube-api-access-nsp8f\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.266380 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.266321 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56afff30-8548-4dbc-afee-61ee544dfe22-istio-token\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.269427 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.269383 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/56afff30-8548-4dbc-afee-61ee544dfe22-cacerts\") pod \"istiod-openshift-gateway-7574459656-7dh8j\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.332877 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.332845 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:54.860879 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.860838 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39f2d9d3-4b27-4b65-9725-7b30babbe5e5-tls-certs\") pod \"model-serving-api-54d89bd5fc-xgwlw\" (UID: \"39f2d9d3-4b27-4b65-9725-7b30babbe5e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-xgwlw" Jul 16 04:44:54.866493 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.866433 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39f2d9d3-4b27-4b65-9725-7b30babbe5e5-tls-certs\") pod \"model-serving-api-54d89bd5fc-xgwlw\" (UID: \"39f2d9d3-4b27-4b65-9725-7b30babbe5e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-xgwlw" Jul 16 04:44:54.976935 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:54.976871 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-xgwlw" Jul 16 04:44:55.389800 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:55.389768 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:44:56.418157 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:56.418124 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xgwlw"] Jul 16 04:44:56.423310 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:44:56.423282 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39f2d9d3_4b27_4b65_9725_7b30babbe5e5.slice/crio-e45e596881a8c33ce043b722a9c3f51806a1c7aca13746c61ba1f688f33c999c WatchSource:0}: Error finding container e45e596881a8c33ce043b722a9c3f51806a1c7aca13746c61ba1f688f33c999c: Status 404 returned error can't find the container with id e45e596881a8c33ce043b722a9c3f51806a1c7aca13746c61ba1f688f33c999c Jul 16 04:44:56.447190 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:56.447126 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j"] Jul 16 04:44:56.450629 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:44:56.450598 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56afff30_8548_4dbc_afee_61ee544dfe22.slice/crio-f9ad18b340a75a67a215c2e5e750c08fc57a5a2c6aab08629c4da6acf7335cd5 WatchSource:0}: Error finding container f9ad18b340a75a67a215c2e5e750c08fc57a5a2c6aab08629c4da6acf7335cd5: Status 404 returned error can't find the container with id f9ad18b340a75a67a215c2e5e750c08fc57a5a2c6aab08629c4da6acf7335cd5 Jul 16 04:44:56.496866 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:56.496792 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" event={"ID":"56afff30-8548-4dbc-afee-61ee544dfe22","Type":"ContainerStarted","Data":"f9ad18b340a75a67a215c2e5e750c08fc57a5a2c6aab08629c4da6acf7335cd5"} Jul 16 04:44:56.497966 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:56.497939 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-xgwlw" event={"ID":"39f2d9d3-4b27-4b65-9725-7b30babbe5e5","Type":"ContainerStarted","Data":"e45e596881a8c33ce043b722a9c3f51806a1c7aca13746c61ba1f688f33c999c"} Jul 16 04:44:56.499329 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:56.499303 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68764b5749-fddrb" event={"ID":"9351e506-d7ac-4be8-a5dc-84f1d25515ef","Type":"ContainerStarted","Data":"43b5e00c5a545b3522198c01d61a28eb5d86a10e8ddf18037ff604d52dc4f361"} Jul 16 04:44:56.500799 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:56.500774 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2q482" event={"ID":"4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5","Type":"ContainerStarted","Data":"6be36ff9deb001d331cbaa7ae46ac68f6d50cf36e940877187215704be8cc584"} Jul 16 04:44:56.500921 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:56.500906 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-2q482" Jul 16 04:44:56.517805 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:56.517765 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-68764b5749-fddrb" podStartSLOduration=1.328460587 podStartE2EDuration="5.517752278s" totalImagesPullingTime="4.189291691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:44:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:44:56.517337163 +0000 UTC m=+349.242805686" watchObservedRunningTime="2026-07-16 04:44:56.517752278 +0000 UTC m=+349.243220800" Jul 16 04:44:56.532573 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:56.532528 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-2q482" podStartSLOduration=2.208047474 podStartE2EDuration="5.532488835s" totalImagesPullingTime="3.324441361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:44:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:44:56.532343509 +0000 UTC m=+349.257812034" watchObservedRunningTime="2026-07-16 04:44:56.532488835 +0000 UTC m=+349.257957358" Jul 16 04:44:59.208535 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:59.208461 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 04:44:59.208810 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:59.208587 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 04:44:59.516872 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:59.516836 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" event={"ID":"56afff30-8548-4dbc-afee-61ee544dfe22","Type":"ContainerStarted","Data":"926b4f276c2a20dcfd0e417f4948606b7d985489237ce4e55e2af6aa9693d38a"} Jul 16 04:44:59.517052 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:59.516958 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:44:59.518432 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:59.518399 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-xgwlw" event={"ID":"39f2d9d3-4b27-4b65-9725-7b30babbe5e5","Type":"ContainerStarted","Data":"fcfc019c4d9146701bf53607eaa26432e1b8bbef6ac62c843ebd38352961deb2"} Jul 16 04:44:59.518549 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:59.518454 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-xgwlw" Jul 16 04:44:59.609116 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:59.609056 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" podStartSLOduration=3.852818563 podStartE2EDuration="6.60904111s" totalImagesPullingTime="2.756222547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:44:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:44:59.555978031 +0000 UTC m=+352.281446553" watchObservedRunningTime="2026-07-16 04:44:59.60904111 +0000 UTC m=+352.334509633" Jul 16 04:44:59.609279 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:44:59.609255 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-xgwlw" podStartSLOduration=3.8262762070000003 podStartE2EDuration="6.609250351s" totalImagesPullingTime="2.782974144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:44:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:44:59.599016735 +0000 UTC m=+352.324485258" watchObservedRunningTime="2026-07-16 04:44:59.609250351 +0000 UTC m=+352.334718873" Jul 16 04:45:00.072964 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:00.072890 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:45:00.463224 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:00.463182 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:45:00.525588 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:00.525557 2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-7dh8j container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 04:45:00.525742 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:00.525616 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" podUID="56afff30-8548-4dbc-afee-61ee544dfe22" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 04:45:01.141527 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.141483 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk"] Jul 16 04:45:01.144623 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.144600 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.157539 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.157498 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-rdzpz\"" Jul 16 04:45:01.193298 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.193272 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk"] Jul 16 04:45:01.217338 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.217311 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.217453 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.217342 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.217453 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.217364 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.217453 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.217415 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.217607 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.217493 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.217607 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.217535 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.217607 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.217559 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.217607 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.217577 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.217743 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.217625 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkvq5\" (UniqueName: \"kubernetes.io/projected/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-kube-api-access-zkvq5\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.318321 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.318284 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.318321 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.318321 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.318603 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.318343 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.318603 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.318393 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.318603 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.318418 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.318603 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.318440 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.318603 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.318477 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.318603 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.318549 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.318603 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.318583 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkvq5\" (UniqueName: \"kubernetes.io/projected/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-kube-api-access-zkvq5\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.318967 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.318940 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.319031 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.318995 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.319098 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.319072 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.319208 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.319180 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.319345 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.319322 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.320934 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.320914 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.321057 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.321039 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.341734 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.341705 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.350832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.350803 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkvq5\" (UniqueName: \"kubernetes.io/projected/89b954af-2c48-4a5e-a06c-6c0bb14cfc69-kube-api-access-zkvq5\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nhbqk\" (UID: \"89b954af-2c48-4a5e-a06c-6c0bb14cfc69\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.395252 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.395163 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:45:01.454987 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.454718 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:01.561142 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.561111 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:45:01.621405 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.621383 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk"] Jul 16 04:45:01.622747 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:45:01.622724 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89b954af_2c48_4a5e_a06c_6c0bb14cfc69.slice/crio-909cdd5456f1357312710ab0accc3dbad01aa0f7ce3d3dfb929e9e942d441cf7 WatchSource:0}: Error finding container 909cdd5456f1357312710ab0accc3dbad01aa0f7ce3d3dfb929e9e942d441cf7: Status 404 returned error can't find the container with id 909cdd5456f1357312710ab0accc3dbad01aa0f7ce3d3dfb929e9e942d441cf7 Jul 16 04:45:01.824937 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:01.824902 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:45:02.534637 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:02.534600 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" event={"ID":"89b954af-2c48-4a5e-a06c-6c0bb14cfc69","Type":"ContainerStarted","Data":"909cdd5456f1357312710ab0accc3dbad01aa0f7ce3d3dfb929e9e942d441cf7"} Jul 16 04:45:03.525140 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:03.525109 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:45:04.647271 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:04.647216 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:45:04.806003 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:04.805945 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:45:04.826733 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:04.826091 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w7tqh"] Jul 16 04:45:04.837298 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:04.837263 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w7tqh" Jul 16 04:45:04.839280 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:04.839097 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7hpfm"] Jul 16 04:45:04.840557 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:04.840479 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 16 04:45:04.842056 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:04.842034 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-bxn94\"" Jul 16 04:45:04.844242 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:04.844222 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7hpfm" Jul 16 04:45:04.847866 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:04.847834 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 16 04:45:04.848202 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:04.848186 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-89ws8\"" Jul 16 04:45:04.854230 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:04.854170 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w7tqh"] Jul 16 04:45:04.866321 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:04.859924 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7hpfm"] Jul 16 04:45:04.940598 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:04.940497 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:45:04.948248 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:04.948218 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwftj\" (UniqueName: \"kubernetes.io/projected/1c73a1b4-1f36-45bb-a3f0-882ee8ef4a50-kube-api-access-xwftj\") pod \"kserve-controller-manager-d6f4b65b8-w7tqh\" (UID: \"1c73a1b4-1f36-45bb-a3f0-882ee8ef4a50\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w7tqh" Jul 16 04:45:04.948375 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:04.948319 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkft5\" (UniqueName: \"kubernetes.io/projected/e438ea2c-9d44-4919-b90f-6f7ef2d86612-kube-api-access-wkft5\") pod \"llmisvc-controller-manager-777c6bd7d5-7hpfm\" (UID: \"e438ea2c-9d44-4919-b90f-6f7ef2d86612\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7hpfm" Jul 16 04:45:04.948375 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:04.948353 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c73a1b4-1f36-45bb-a3f0-882ee8ef4a50-cert\") pod \"kserve-controller-manager-d6f4b65b8-w7tqh\" (UID: \"1c73a1b4-1f36-45bb-a3f0-882ee8ef4a50\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w7tqh" Jul 16 04:45:04.948478 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:04.948390 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e438ea2c-9d44-4919-b90f-6f7ef2d86612-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7hpfm\" (UID: \"e438ea2c-9d44-4919-b90f-6f7ef2d86612\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7hpfm" Jul 16 04:45:05.006123 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.006090 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 04:45:05.006389 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.006157 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 04:45:05.006389 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.006188 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 04:45:05.049590 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.049553 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwftj\" (UniqueName: \"kubernetes.io/projected/1c73a1b4-1f36-45bb-a3f0-882ee8ef4a50-kube-api-access-xwftj\") pod \"kserve-controller-manager-d6f4b65b8-w7tqh\" (UID: \"1c73a1b4-1f36-45bb-a3f0-882ee8ef4a50\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w7tqh" Jul 16 04:45:05.049726 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.049679 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkft5\" (UniqueName: \"kubernetes.io/projected/e438ea2c-9d44-4919-b90f-6f7ef2d86612-kube-api-access-wkft5\") pod \"llmisvc-controller-manager-777c6bd7d5-7hpfm\" (UID: \"e438ea2c-9d44-4919-b90f-6f7ef2d86612\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7hpfm" Jul 16 04:45:05.049726 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.049717 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c73a1b4-1f36-45bb-a3f0-882ee8ef4a50-cert\") pod \"kserve-controller-manager-d6f4b65b8-w7tqh\" (UID: \"1c73a1b4-1f36-45bb-a3f0-882ee8ef4a50\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w7tqh" Jul 16 04:45:05.049857 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.049748 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e438ea2c-9d44-4919-b90f-6f7ef2d86612-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7hpfm\" (UID: \"e438ea2c-9d44-4919-b90f-6f7ef2d86612\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7hpfm" Jul 16 04:45:05.049948 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:45:05.049938 2581 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 16 04:45:05.050446 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:45:05.050041 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e438ea2c-9d44-4919-b90f-6f7ef2d86612-cert podName:e438ea2c-9d44-4919-b90f-6f7ef2d86612 nodeName:}" failed. No retries permitted until 2026-07-16 04:45:05.55001775 +0000 UTC m=+358.275486263 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e438ea2c-9d44-4919-b90f-6f7ef2d86612-cert") pod "llmisvc-controller-manager-777c6bd7d5-7hpfm" (UID: "e438ea2c-9d44-4919-b90f-6f7ef2d86612") : secret "llmisvc-webhook-server-cert" not found Jul 16 04:45:05.052557 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.052534 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c73a1b4-1f36-45bb-a3f0-882ee8ef4a50-cert\") pod \"kserve-controller-manager-d6f4b65b8-w7tqh\" (UID: \"1c73a1b4-1f36-45bb-a3f0-882ee8ef4a50\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w7tqh" Jul 16 04:45:05.059439 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.059409 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkft5\" (UniqueName: \"kubernetes.io/projected/e438ea2c-9d44-4919-b90f-6f7ef2d86612-kube-api-access-wkft5\") pod \"llmisvc-controller-manager-777c6bd7d5-7hpfm\" (UID: \"e438ea2c-9d44-4919-b90f-6f7ef2d86612\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7hpfm" Jul 16 04:45:05.063829 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.063811 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwftj\" (UniqueName: \"kubernetes.io/projected/1c73a1b4-1f36-45bb-a3f0-882ee8ef4a50-kube-api-access-xwftj\") pod \"kserve-controller-manager-d6f4b65b8-w7tqh\" (UID: \"1c73a1b4-1f36-45bb-a3f0-882ee8ef4a50\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w7tqh" Jul 16 04:45:05.113834 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.113804 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:45:05.152485 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.152442 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w7tqh" Jul 16 04:45:05.311707 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.311682 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w7tqh"] Jul 16 04:45:05.314579 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:45:05.314549 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c73a1b4_1f36_45bb_a3f0_882ee8ef4a50.slice/crio-854d2d5e87c77649fc8fbfe17ea2bbdfa2308e1498a1131aeabe5f37cf47e8ae WatchSource:0}: Error finding container 854d2d5e87c77649fc8fbfe17ea2bbdfa2308e1498a1131aeabe5f37cf47e8ae: Status 404 returned error can't find the container with id 854d2d5e87c77649fc8fbfe17ea2bbdfa2308e1498a1131aeabe5f37cf47e8ae Jul 16 04:45:05.370667 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.370640 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:45:05.550428 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.550347 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" event={"ID":"89b954af-2c48-4a5e-a06c-6c0bb14cfc69","Type":"ContainerStarted","Data":"bfe9e44710cea09764d336c84e4258b704428fdf18f6968530432b7f812b1712"} Jul 16 04:45:05.551626 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.551604 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w7tqh" event={"ID":"1c73a1b4-1f36-45bb-a3f0-882ee8ef4a50","Type":"ContainerStarted","Data":"854d2d5e87c77649fc8fbfe17ea2bbdfa2308e1498a1131aeabe5f37cf47e8ae"} Jul 16 04:45:05.554965 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.554942 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e438ea2c-9d44-4919-b90f-6f7ef2d86612-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7hpfm\" (UID: \"e438ea2c-9d44-4919-b90f-6f7ef2d86612\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7hpfm" Jul 16 04:45:05.557155 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.557139 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e438ea2c-9d44-4919-b90f-6f7ef2d86612-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7hpfm\" (UID: \"e438ea2c-9d44-4919-b90f-6f7ef2d86612\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7hpfm" Jul 16 04:45:05.766812 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.766776 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7hpfm" Jul 16 04:45:05.906436 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.906394 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" podStartSLOduration=1.525245135 podStartE2EDuration="4.906376201s" totalImagesPullingTime="3.381131066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:45:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:45:05.593334201 +0000 UTC m=+358.318802761" watchObservedRunningTime="2026-07-16 04:45:05.906376201 +0000 UTC m=+358.631844722" Jul 16 04:45:05.906885 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.906868 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7hpfm"] Jul 16 04:45:05.908559 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:45:05.908531 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode438ea2c_9d44_4919_b90f_6f7ef2d86612.slice/crio-0a2c322f673109db0c9d9a266a12b1002be8f03f6f6c302c8115db19c2f115e4 WatchSource:0}: Error finding container 0a2c322f673109db0c9d9a266a12b1002be8f03f6f6c302c8115db19c2f115e4: Status 404 returned error can't find the container with id 0a2c322f673109db0c9d9a266a12b1002be8f03f6f6c302c8115db19c2f115e4 Jul 16 04:45:05.926340 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.926317 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j"] Jul 16 04:45:05.926490 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:05.926470 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" podUID="56afff30-8548-4dbc-afee-61ee544dfe22" containerName="discovery" containerID="cri-o://926b4f276c2a20dcfd0e417f4948606b7d985489237ce4e55e2af6aa9693d38a" gracePeriod=30 Jul 16 04:45:06.096659 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.096576 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:45:06.186149 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.186114 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:45:06.191843 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.191791 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:45:06.261333 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.261296 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/56afff30-8548-4dbc-afee-61ee544dfe22-istio-csr-dns-cert\") pod \"56afff30-8548-4dbc-afee-61ee544dfe22\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " Jul 16 04:45:06.261537 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.261393 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/56afff30-8548-4dbc-afee-61ee544dfe22-cacerts\") pod \"56afff30-8548-4dbc-afee-61ee544dfe22\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " Jul 16 04:45:06.261537 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.261433 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/56afff30-8548-4dbc-afee-61ee544dfe22-istio-csr-ca-configmap\") pod \"56afff30-8548-4dbc-afee-61ee544dfe22\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " Jul 16 04:45:06.261537 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.261480 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56afff30-8548-4dbc-afee-61ee544dfe22-istio-token\") pod \"56afff30-8548-4dbc-afee-61ee544dfe22\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " Jul 16 04:45:06.261710 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.261535 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/56afff30-8548-4dbc-afee-61ee544dfe22-local-certs\") pod \"56afff30-8548-4dbc-afee-61ee544dfe22\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " Jul 16 04:45:06.261710 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.261567 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56afff30-8548-4dbc-afee-61ee544dfe22-istio-kubeconfig\") pod \"56afff30-8548-4dbc-afee-61ee544dfe22\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " Jul 16 04:45:06.261710 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.261603 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsp8f\" (UniqueName: \"kubernetes.io/projected/56afff30-8548-4dbc-afee-61ee544dfe22-kube-api-access-nsp8f\") pod \"56afff30-8548-4dbc-afee-61ee544dfe22\" (UID: \"56afff30-8548-4dbc-afee-61ee544dfe22\") " Jul 16 04:45:06.261922 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.261894 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56afff30-8548-4dbc-afee-61ee544dfe22-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "56afff30-8548-4dbc-afee-61ee544dfe22" (UID: "56afff30-8548-4dbc-afee-61ee544dfe22"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:45:06.264279 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.264248 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56afff30-8548-4dbc-afee-61ee544dfe22-cacerts" (OuterVolumeSpecName: "cacerts") pod "56afff30-8548-4dbc-afee-61ee544dfe22" (UID: "56afff30-8548-4dbc-afee-61ee544dfe22"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:45:06.264493 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.264463 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56afff30-8548-4dbc-afee-61ee544dfe22-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "56afff30-8548-4dbc-afee-61ee544dfe22" (UID: "56afff30-8548-4dbc-afee-61ee544dfe22"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:45:06.264915 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.264877 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56afff30-8548-4dbc-afee-61ee544dfe22-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "56afff30-8548-4dbc-afee-61ee544dfe22" (UID: "56afff30-8548-4dbc-afee-61ee544dfe22"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:45:06.265192 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.265135 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56afff30-8548-4dbc-afee-61ee544dfe22-istio-token" (OuterVolumeSpecName: "istio-token") pod "56afff30-8548-4dbc-afee-61ee544dfe22" (UID: "56afff30-8548-4dbc-afee-61ee544dfe22"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:45:06.265192 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.265158 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56afff30-8548-4dbc-afee-61ee544dfe22-kube-api-access-nsp8f" (OuterVolumeSpecName: "kube-api-access-nsp8f") pod "56afff30-8548-4dbc-afee-61ee544dfe22" (UID: "56afff30-8548-4dbc-afee-61ee544dfe22"). InnerVolumeSpecName "kube-api-access-nsp8f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:45:06.265309 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.265202 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56afff30-8548-4dbc-afee-61ee544dfe22-local-certs" (OuterVolumeSpecName: "local-certs") pod "56afff30-8548-4dbc-afee-61ee544dfe22" (UID: "56afff30-8548-4dbc-afee-61ee544dfe22"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:45:06.362335 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.362252 2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/56afff30-8548-4dbc-afee-61ee544dfe22-istio-csr-ca-configmap\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:45:06.362335 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.362288 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56afff30-8548-4dbc-afee-61ee544dfe22-istio-token\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:45:06.362335 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.362301 2581 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/56afff30-8548-4dbc-afee-61ee544dfe22-local-certs\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:45:06.362335 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.362312 2581 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56afff30-8548-4dbc-afee-61ee544dfe22-istio-kubeconfig\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:45:06.362335 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.362320 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsp8f\" (UniqueName: \"kubernetes.io/projected/56afff30-8548-4dbc-afee-61ee544dfe22-kube-api-access-nsp8f\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:45:06.362335 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.362328 2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/56afff30-8548-4dbc-afee-61ee544dfe22-istio-csr-dns-cert\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:45:06.362335 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.362340 2581 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/56afff30-8548-4dbc-afee-61ee544dfe22-cacerts\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:45:06.456185 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.456150 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:06.461675 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.461644 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:06.558094 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.558064 2581 generic.go:356] "Generic (PLEG): container finished" podID="56afff30-8548-4dbc-afee-61ee544dfe22" containerID="926b4f276c2a20dcfd0e417f4948606b7d985489237ce4e55e2af6aa9693d38a" exitCode=0 Jul 16 04:45:06.558281 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.558121 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" Jul 16 04:45:06.558281 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.558126 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" event={"ID":"56afff30-8548-4dbc-afee-61ee544dfe22","Type":"ContainerDied","Data":"926b4f276c2a20dcfd0e417f4948606b7d985489237ce4e55e2af6aa9693d38a"} Jul 16 04:45:06.558281 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.558182 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j" event={"ID":"56afff30-8548-4dbc-afee-61ee544dfe22","Type":"ContainerDied","Data":"f9ad18b340a75a67a215c2e5e750c08fc57a5a2c6aab08629c4da6acf7335cd5"} Jul 16 04:45:06.558281 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.558199 2581 scope.go:117] "RemoveContainer" containerID="926b4f276c2a20dcfd0e417f4948606b7d985489237ce4e55e2af6aa9693d38a" Jul 16 04:45:06.559794 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.559768 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7hpfm" event={"ID":"e438ea2c-9d44-4919-b90f-6f7ef2d86612","Type":"ContainerStarted","Data":"0a2c322f673109db0c9d9a266a12b1002be8f03f6f6c302c8115db19c2f115e4"} Jul 16 04:45:06.560273 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.560200 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:06.561554 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.561529 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nhbqk" Jul 16 04:45:06.574937 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.574914 2581 scope.go:117] "RemoveContainer" containerID="926b4f276c2a20dcfd0e417f4948606b7d985489237ce4e55e2af6aa9693d38a" Jul 16 04:45:06.576955 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:45:06.576924 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"926b4f276c2a20dcfd0e417f4948606b7d985489237ce4e55e2af6aa9693d38a\": container with ID starting with 926b4f276c2a20dcfd0e417f4948606b7d985489237ce4e55e2af6aa9693d38a not found: ID does not exist" containerID="926b4f276c2a20dcfd0e417f4948606b7d985489237ce4e55e2af6aa9693d38a" Jul 16 04:45:06.577059 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.576965 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"926b4f276c2a20dcfd0e417f4948606b7d985489237ce4e55e2af6aa9693d38a"} err="failed to get container status \"926b4f276c2a20dcfd0e417f4948606b7d985489237ce4e55e2af6aa9693d38a\": rpc error: code = NotFound desc = could not find container \"926b4f276c2a20dcfd0e417f4948606b7d985489237ce4e55e2af6aa9693d38a\": container with ID starting with 926b4f276c2a20dcfd0e417f4948606b7d985489237ce4e55e2af6aa9693d38a not found: ID does not exist" Jul 16 04:45:06.626118 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.625987 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j"] Jul 16 04:45:06.630837 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:06.630786 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7dh8j"] Jul 16 04:45:07.509246 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:07.509146 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-2q482" Jul 16 04:45:07.839540 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:07.839455 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56afff30-8548-4dbc-afee-61ee544dfe22" path="/var/lib/kubelet/pods/56afff30-8548-4dbc-afee-61ee544dfe22/volumes" Jul 16 04:45:08.573709 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:08.573677 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w7tqh" event={"ID":"1c73a1b4-1f36-45bb-a3f0-882ee8ef4a50","Type":"ContainerStarted","Data":"534025bc61be85ad1d11ef2299696aa67a4781ea01dddd187a2791c70c2e521f"} Jul 16 04:45:08.632615 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:08.632568 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w7tqh" podStartSLOduration=1.933315494 podStartE2EDuration="4.632552072s" totalImagesPullingTime="2.699236578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:45:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:45:08.631427555 +0000 UTC m=+361.356896093" watchObservedRunningTime="2026-07-16 04:45:08.632552072 +0000 UTC m=+361.358020593" Jul 16 04:45:09.582891 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:09.582844 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7hpfm" event={"ID":"e438ea2c-9d44-4919-b90f-6f7ef2d86612","Type":"ContainerStarted","Data":"b36c1675739d4df96cac0577b1cb81ca6e86113f38b0d15957b210437baf18b2"} Jul 16 04:45:09.583655 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:09.582912 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w7tqh" Jul 16 04:45:09.583655 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:09.582965 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7hpfm" Jul 16 04:45:09.619010 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:09.615185 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7hpfm" podStartSLOduration=2.409366814 podStartE2EDuration="5.615166851s" totalImagesPullingTime="3.205800037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:45:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:45:09.614187764 +0000 UTC m=+362.339656287" watchObservedRunningTime="2026-07-16 04:45:09.615166851 +0000 UTC m=+362.340635377" Jul 16 04:45:10.527559 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:10.527527 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-xgwlw" Jul 16 04:45:20.459863 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:20.459831 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:45:20.558577 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:20.558520 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:45:40.588622 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:40.588588 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7hpfm" Jul 16 04:45:40.591751 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:40.591731 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w7tqh" Jul 16 04:45:52.322428 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:52.322379 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:45:52.424644 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:52.424608 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:45:52.476548 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:52.476517 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5vw4q"] Jul 16 04:45:52.476861 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:52.476824 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56afff30-8548-4dbc-afee-61ee544dfe22" containerName="discovery" Jul 16 04:45:52.476861 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:52.476837 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="56afff30-8548-4dbc-afee-61ee544dfe22" containerName="discovery" Jul 16 04:45:52.476972 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:52.476905 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="56afff30-8548-4dbc-afee-61ee544dfe22" containerName="discovery" Jul 16 04:45:52.484337 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:52.484304 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5vw4q" Jul 16 04:45:52.487407 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:52.487377 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 04:45:52.488405 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:52.488385 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 04:45:52.488556 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:52.488407 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-496rg\"" Jul 16 04:45:52.492328 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:52.492299 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5vw4q"] Jul 16 04:45:52.541272 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:52.541222 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:45:52.552795 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:52.552767 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtl4r\" (UniqueName: \"kubernetes.io/projected/417d930c-22b6-4829-86fa-7cd6b926b141-kube-api-access-jtl4r\") pod \"kuadrant-operator-catalog-5vw4q\" (UID: \"417d930c-22b6-4829-86fa-7cd6b926b141\") " pod="kuadrant-system/kuadrant-operator-catalog-5vw4q" Jul 16 04:45:52.653784 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:52.653694 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtl4r\" (UniqueName: \"kubernetes.io/projected/417d930c-22b6-4829-86fa-7cd6b926b141-kube-api-access-jtl4r\") pod \"kuadrant-operator-catalog-5vw4q\" (UID: \"417d930c-22b6-4829-86fa-7cd6b926b141\") " pod="kuadrant-system/kuadrant-operator-catalog-5vw4q" Jul 16 04:45:52.662696 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:52.662671 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtl4r\" (UniqueName: \"kubernetes.io/projected/417d930c-22b6-4829-86fa-7cd6b926b141-kube-api-access-jtl4r\") pod \"kuadrant-operator-catalog-5vw4q\" (UID: \"417d930c-22b6-4829-86fa-7cd6b926b141\") " pod="kuadrant-system/kuadrant-operator-catalog-5vw4q" Jul 16 04:45:52.795261 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:52.795235 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5vw4q" Jul 16 04:45:52.823291 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:52.823266 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5vw4q"] Jul 16 04:45:52.926197 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:52.926170 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5vw4q"] Jul 16 04:45:52.928664 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:45:52.928633 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod417d930c_22b6_4829_86fa_7cd6b926b141.slice/crio-3a0907b99c70dbd85574fe9d63f61f29ebf1b7d331ac67e50b78d68e63785c39 WatchSource:0}: Error finding container 3a0907b99c70dbd85574fe9d63f61f29ebf1b7d331ac67e50b78d68e63785c39: Status 404 returned error can't find the container with id 3a0907b99c70dbd85574fe9d63f61f29ebf1b7d331ac67e50b78d68e63785c39 Jul 16 04:45:53.037245 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:53.037210 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tvvv9"] Jul 16 04:45:53.040383 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:53.040365 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tvvv9" Jul 16 04:45:53.048401 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:53.048380 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tvvv9"] Jul 16 04:45:53.157914 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:53.157885 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75ckl\" (UniqueName: \"kubernetes.io/projected/f99d39f4-5750-4fab-bef8-a8cd7f2e9e6f-kube-api-access-75ckl\") pod \"kuadrant-operator-catalog-tvvv9\" (UID: \"f99d39f4-5750-4fab-bef8-a8cd7f2e9e6f\") " pod="kuadrant-system/kuadrant-operator-catalog-tvvv9" Jul 16 04:45:53.258630 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:53.258598 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75ckl\" (UniqueName: \"kubernetes.io/projected/f99d39f4-5750-4fab-bef8-a8cd7f2e9e6f-kube-api-access-75ckl\") pod \"kuadrant-operator-catalog-tvvv9\" (UID: \"f99d39f4-5750-4fab-bef8-a8cd7f2e9e6f\") " pod="kuadrant-system/kuadrant-operator-catalog-tvvv9" Jul 16 04:45:53.269301 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:53.269275 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75ckl\" (UniqueName: \"kubernetes.io/projected/f99d39f4-5750-4fab-bef8-a8cd7f2e9e6f-kube-api-access-75ckl\") pod \"kuadrant-operator-catalog-tvvv9\" (UID: \"f99d39f4-5750-4fab-bef8-a8cd7f2e9e6f\") " pod="kuadrant-system/kuadrant-operator-catalog-tvvv9" Jul 16 04:45:53.350296 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:53.350267 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tvvv9" Jul 16 04:45:53.477834 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:53.477812 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tvvv9"] Jul 16 04:45:53.480083 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:45:53.480054 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf99d39f4_5750_4fab_bef8_a8cd7f2e9e6f.slice/crio-59c1d6d59fdf336c041ffe0e14dbdaa8f0fca2cc3aaed8679db1f4464feb5b18 WatchSource:0}: Error finding container 59c1d6d59fdf336c041ffe0e14dbdaa8f0fca2cc3aaed8679db1f4464feb5b18: Status 404 returned error can't find the container with id 59c1d6d59fdf336c041ffe0e14dbdaa8f0fca2cc3aaed8679db1f4464feb5b18 Jul 16 04:45:53.769647 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:53.769610 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tvvv9" event={"ID":"f99d39f4-5750-4fab-bef8-a8cd7f2e9e6f","Type":"ContainerStarted","Data":"59c1d6d59fdf336c041ffe0e14dbdaa8f0fca2cc3aaed8679db1f4464feb5b18"} Jul 16 04:45:53.770798 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:53.770768 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5vw4q" event={"ID":"417d930c-22b6-4829-86fa-7cd6b926b141","Type":"ContainerStarted","Data":"3a0907b99c70dbd85574fe9d63f61f29ebf1b7d331ac67e50b78d68e63785c39"} Jul 16 04:45:55.782328 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:55.782290 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tvvv9" event={"ID":"f99d39f4-5750-4fab-bef8-a8cd7f2e9e6f","Type":"ContainerStarted","Data":"5baae9edcb19a1e284830b9028e3c685130abc5d0d9a278efc7439a2cddacedd"} Jul 16 04:45:55.783880 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:55.783851 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5vw4q" event={"ID":"417d930c-22b6-4829-86fa-7cd6b926b141","Type":"ContainerStarted","Data":"432068cad4a8ac33b50a0e1c4ecdbcec25aadb69afec81d4c288d34ab3477e69"} Jul 16 04:45:55.783880 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:55.783865 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-5vw4q" podUID="417d930c-22b6-4829-86fa-7cd6b926b141" containerName="registry-server" containerID="cri-o://432068cad4a8ac33b50a0e1c4ecdbcec25aadb69afec81d4c288d34ab3477e69" gracePeriod=2 Jul 16 04:45:55.806745 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:55.806702 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-tvvv9" podStartSLOduration=1.35833454 podStartE2EDuration="2.806688899s" totalImagesPullingTime="1.448354359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:45:55.805728881 +0000 UTC m=+408.531197403" watchObservedRunningTime="2026-07-16 04:45:55.806688899 +0000 UTC m=+408.532157422" Jul 16 04:45:55.825638 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:55.825588 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5vw4q" podStartSLOduration=1.836376848 podStartE2EDuration="3.825571322s" totalImagesPullingTime="1.989194474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:45:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:45:55.824011759 +0000 UTC m=+408.549480310" watchObservedRunningTime="2026-07-16 04:45:55.825571322 +0000 UTC m=+408.551039843" Jul 16 04:45:56.030399 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:56.030375 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5vw4q" Jul 16 04:45:56.183630 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:56.183551 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtl4r\" (UniqueName: \"kubernetes.io/projected/417d930c-22b6-4829-86fa-7cd6b926b141-kube-api-access-jtl4r\") pod \"417d930c-22b6-4829-86fa-7cd6b926b141\" (UID: \"417d930c-22b6-4829-86fa-7cd6b926b141\") " Jul 16 04:45:56.185758 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:56.185736 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/417d930c-22b6-4829-86fa-7cd6b926b141-kube-api-access-jtl4r" (OuterVolumeSpecName: "kube-api-access-jtl4r") pod "417d930c-22b6-4829-86fa-7cd6b926b141" (UID: "417d930c-22b6-4829-86fa-7cd6b926b141"). InnerVolumeSpecName "kube-api-access-jtl4r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:45:56.284791 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:56.284764 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtl4r\" (UniqueName: \"kubernetes.io/projected/417d930c-22b6-4829-86fa-7cd6b926b141-kube-api-access-jtl4r\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:45:56.789933 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:56.789897 2581 generic.go:356] "Generic (PLEG): container finished" podID="417d930c-22b6-4829-86fa-7cd6b926b141" containerID="432068cad4a8ac33b50a0e1c4ecdbcec25aadb69afec81d4c288d34ab3477e69" exitCode=0 Jul 16 04:45:56.790376 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:56.789944 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5vw4q" Jul 16 04:45:56.790376 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:56.789993 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5vw4q" event={"ID":"417d930c-22b6-4829-86fa-7cd6b926b141","Type":"ContainerDied","Data":"432068cad4a8ac33b50a0e1c4ecdbcec25aadb69afec81d4c288d34ab3477e69"} Jul 16 04:45:56.790376 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:56.790058 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5vw4q" event={"ID":"417d930c-22b6-4829-86fa-7cd6b926b141","Type":"ContainerDied","Data":"3a0907b99c70dbd85574fe9d63f61f29ebf1b7d331ac67e50b78d68e63785c39"} Jul 16 04:45:56.790376 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:56.790102 2581 scope.go:117] "RemoveContainer" containerID="432068cad4a8ac33b50a0e1c4ecdbcec25aadb69afec81d4c288d34ab3477e69" Jul 16 04:45:56.799490 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:56.799469 2581 scope.go:117] "RemoveContainer" containerID="432068cad4a8ac33b50a0e1c4ecdbcec25aadb69afec81d4c288d34ab3477e69" Jul 16 04:45:56.799762 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:45:56.799745 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"432068cad4a8ac33b50a0e1c4ecdbcec25aadb69afec81d4c288d34ab3477e69\": container with ID starting with 432068cad4a8ac33b50a0e1c4ecdbcec25aadb69afec81d4c288d34ab3477e69 not found: ID does not exist" containerID="432068cad4a8ac33b50a0e1c4ecdbcec25aadb69afec81d4c288d34ab3477e69" Jul 16 04:45:56.799808 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:56.799771 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"432068cad4a8ac33b50a0e1c4ecdbcec25aadb69afec81d4c288d34ab3477e69"} err="failed to get container status \"432068cad4a8ac33b50a0e1c4ecdbcec25aadb69afec81d4c288d34ab3477e69\": rpc error: code = NotFound desc = could not find container \"432068cad4a8ac33b50a0e1c4ecdbcec25aadb69afec81d4c288d34ab3477e69\": container with ID starting with 432068cad4a8ac33b50a0e1c4ecdbcec25aadb69afec81d4c288d34ab3477e69 not found: ID does not exist" Jul 16 04:45:56.811262 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:56.811242 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5vw4q"] Jul 16 04:45:56.815607 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:56.815587 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5vw4q"] Jul 16 04:45:57.837881 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:45:57.837838 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="417d930c-22b6-4829-86fa-7cd6b926b141" path="/var/lib/kubelet/pods/417d930c-22b6-4829-86fa-7cd6b926b141/volumes" Jul 16 04:46:03.350801 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:03.350769 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-tvvv9" Jul 16 04:46:03.351202 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:03.350856 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-tvvv9" Jul 16 04:46:03.372857 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:03.372833 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-tvvv9" Jul 16 04:46:03.848914 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:03.848888 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-tvvv9" Jul 16 04:46:18.078913 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:18.078883 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:18.131355 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:18.131330 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:21.033535 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:21.033448 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:24.468841 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:24.468808 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:25.813837 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:25.813809 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94"] Jul 16 04:46:25.814187 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:25.814106 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="417d930c-22b6-4829-86fa-7cd6b926b141" containerName="registry-server" Jul 16 04:46:25.814187 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:25.814116 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="417d930c-22b6-4829-86fa-7cd6b926b141" containerName="registry-server" Jul 16 04:46:25.814187 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:25.814176 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="417d930c-22b6-4829-86fa-7cd6b926b141" containerName="registry-server" Jul 16 04:46:25.818437 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:25.818420 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:25.821499 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:25.821481 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dgrpj\"" Jul 16 04:46:25.839532 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:25.839491 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94"] Jul 16 04:46:25.915480 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:25.915453 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3db2226b-912a-4512-967f-8a2c97793ebc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k5t94\" (UID: \"3db2226b-912a-4512-967f-8a2c97793ebc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:25.915682 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:25.915664 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnrrh\" (UniqueName: \"kubernetes.io/projected/3db2226b-912a-4512-967f-8a2c97793ebc-kube-api-access-tnrrh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k5t94\" (UID: \"3db2226b-912a-4512-967f-8a2c97793ebc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:26.016852 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:26.016826 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnrrh\" (UniqueName: \"kubernetes.io/projected/3db2226b-912a-4512-967f-8a2c97793ebc-kube-api-access-tnrrh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k5t94\" (UID: \"3db2226b-912a-4512-967f-8a2c97793ebc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:26.016954 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:26.016868 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3db2226b-912a-4512-967f-8a2c97793ebc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k5t94\" (UID: \"3db2226b-912a-4512-967f-8a2c97793ebc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:26.017186 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:26.017171 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3db2226b-912a-4512-967f-8a2c97793ebc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k5t94\" (UID: \"3db2226b-912a-4512-967f-8a2c97793ebc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:26.026961 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:26.026942 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnrrh\" (UniqueName: \"kubernetes.io/projected/3db2226b-912a-4512-967f-8a2c97793ebc-kube-api-access-tnrrh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k5t94\" (UID: \"3db2226b-912a-4512-967f-8a2c97793ebc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:26.129049 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:26.128983 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:26.262496 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:26.262466 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94"] Jul 16 04:46:26.265148 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:46:26.265120 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3db2226b_912a_4512_967f_8a2c97793ebc.slice/crio-e299fb79debd79049f4be0c6e4e71e1aeef79913593d428e4a4076d1f9558651 WatchSource:0}: Error finding container e299fb79debd79049f4be0c6e4e71e1aeef79913593d428e4a4076d1f9558651: Status 404 returned error can't find the container with id e299fb79debd79049f4be0c6e4e71e1aeef79913593d428e4a4076d1f9558651 Jul 16 04:46:26.445387 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:26.445294 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:26.918140 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:26.918091 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" event={"ID":"3db2226b-912a-4512-967f-8a2c97793ebc","Type":"ContainerStarted","Data":"e299fb79debd79049f4be0c6e4e71e1aeef79913593d428e4a4076d1f9558651"} Jul 16 04:46:27.756985 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:27.756945 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-s8f5d"] Jul 16 04:46:27.763683 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:27.763648 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s8f5d" Jul 16 04:46:27.766520 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:27.766489 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-fzrtn\"" Jul 16 04:46:27.766846 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:27.766831 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 16 04:46:27.773156 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:27.773131 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-s8f5d"] Jul 16 04:46:27.831050 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:27.831023 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttzw4\" (UniqueName: \"kubernetes.io/projected/2b837d07-e062-4906-93b3-eec0ef7b091b-kube-api-access-ttzw4\") pod \"dns-operator-controller-manager-864dd78c4c-s8f5d\" (UID: \"2b837d07-e062-4906-93b3-eec0ef7b091b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s8f5d" Jul 16 04:46:27.931752 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:27.931714 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttzw4\" (UniqueName: \"kubernetes.io/projected/2b837d07-e062-4906-93b3-eec0ef7b091b-kube-api-access-ttzw4\") pod \"dns-operator-controller-manager-864dd78c4c-s8f5d\" (UID: \"2b837d07-e062-4906-93b3-eec0ef7b091b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s8f5d" Jul 16 04:46:27.943229 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:27.943199 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttzw4\" (UniqueName: \"kubernetes.io/projected/2b837d07-e062-4906-93b3-eec0ef7b091b-kube-api-access-ttzw4\") pod \"dns-operator-controller-manager-864dd78c4c-s8f5d\" (UID: \"2b837d07-e062-4906-93b3-eec0ef7b091b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s8f5d" Jul 16 04:46:28.074451 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:28.074371 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s8f5d" Jul 16 04:46:28.211314 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:28.211285 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-s8f5d"] Jul 16 04:46:28.213816 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:46:28.213788 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b837d07_e062_4906_93b3_eec0ef7b091b.slice/crio-9173ea3579345091efdedf60fb42553ff08abfe3bb3676846e75697aa84ff1ca WatchSource:0}: Error finding container 9173ea3579345091efdedf60fb42553ff08abfe3bb3676846e75697aa84ff1ca: Status 404 returned error can't find the container with id 9173ea3579345091efdedf60fb42553ff08abfe3bb3676846e75697aa84ff1ca Jul 16 04:46:28.297909 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:28.297879 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:28.431019 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:28.430937 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:28.833365 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:28.833320 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:28.927461 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:28.927418 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:28.928764 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:28.928741 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s8f5d" event={"ID":"2b837d07-e062-4906-93b3-eec0ef7b091b","Type":"ContainerStarted","Data":"9173ea3579345091efdedf60fb42553ff08abfe3bb3676846e75697aa84ff1ca"} Jul 16 04:46:29.030629 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:29.030561 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:29.127211 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:29.127127 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:30.657109 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:30.656932 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:30.683639 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:30.683609 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw"] Jul 16 04:46:30.687143 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:30.687120 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" Jul 16 04:46:30.691346 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:30.691305 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-28zk7\"" Jul 16 04:46:30.699578 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:30.699554 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw"] Jul 16 04:46:30.757824 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:30.757796 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwr9v\" (UniqueName: \"kubernetes.io/projected/87a513d6-0edb-467e-831f-6f1673ae9f42-kube-api-access-vwr9v\") pod \"limitador-operator-controller-manager-859d7b78bd-rmtkw\" (UID: \"87a513d6-0edb-467e-831f-6f1673ae9f42\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" Jul 16 04:46:30.807759 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:30.807724 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:30.858682 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:30.858649 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwr9v\" (UniqueName: \"kubernetes.io/projected/87a513d6-0edb-467e-831f-6f1673ae9f42-kube-api-access-vwr9v\") pod \"limitador-operator-controller-manager-859d7b78bd-rmtkw\" (UID: \"87a513d6-0edb-467e-831f-6f1673ae9f42\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" Jul 16 04:46:30.871596 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:30.871559 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwr9v\" (UniqueName: \"kubernetes.io/projected/87a513d6-0edb-467e-831f-6f1673ae9f42-kube-api-access-vwr9v\") pod \"limitador-operator-controller-manager-859d7b78bd-rmtkw\" (UID: \"87a513d6-0edb-467e-831f-6f1673ae9f42\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" Jul 16 04:46:31.001363 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:31.001335 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" Jul 16 04:46:31.609919 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:31.609812 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:31.724974 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:31.724948 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw"] Jul 16 04:46:31.726361 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:46:31.726336 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87a513d6_0edb_467e_831f_6f1673ae9f42.slice/crio-97931de483d16e2452f4d8fd23dcf16d871869303a941b91addcf101cd60a814 WatchSource:0}: Error finding container 97931de483d16e2452f4d8fd23dcf16d871869303a941b91addcf101cd60a814: Status 404 returned error can't find the container with id 97931de483d16e2452f4d8fd23dcf16d871869303a941b91addcf101cd60a814 Jul 16 04:46:31.950272 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:31.950189 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" event={"ID":"3db2226b-912a-4512-967f-8a2c97793ebc","Type":"ContainerStarted","Data":"9ec4f0df53e38c9dd13a87602fd317944e0455ded6cf6b31d7d25fc48f156ba7"} Jul 16 04:46:31.950445 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:31.950306 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:31.952072 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:31.952041 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" event={"ID":"87a513d6-0edb-467e-831f-6f1673ae9f42","Type":"ContainerStarted","Data":"97931de483d16e2452f4d8fd23dcf16d871869303a941b91addcf101cd60a814"} Jul 16 04:46:31.953924 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:31.953899 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s8f5d" event={"ID":"2b837d07-e062-4906-93b3-eec0ef7b091b","Type":"ContainerStarted","Data":"579e77562ae5a9a0c70d6b15bcc1487547a476410c71ed7add6413b0c216f193"} Jul 16 04:46:31.954060 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:31.954044 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s8f5d" Jul 16 04:46:31.982852 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:31.982812 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" podStartSLOduration=1.67102244 podStartE2EDuration="6.982798492s" totalImagesPullingTime="5.311776052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:46:31.981124569 +0000 UTC m=+444.706593109" watchObservedRunningTime="2026-07-16 04:46:31.982798492 +0000 UTC m=+444.708267014" Jul 16 04:46:32.006943 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:32.006903 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s8f5d" podStartSLOduration=1.6618659770000002 podStartE2EDuration="5.006890215s" totalImagesPullingTime="3.345024238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:46:32.005683949 +0000 UTC m=+444.731152472" watchObservedRunningTime="2026-07-16 04:46:32.006890215 +0000 UTC m=+444.732358736" Jul 16 04:46:33.052447 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:33.052382 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:33.166635 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:33.166602 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:33.964889 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:33.964856 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" event={"ID":"87a513d6-0edb-467e-831f-6f1673ae9f42","Type":"ContainerStarted","Data":"904a71d26fb5caf04df44795db1bd3ee076d08eeb127591143e367c7c6a97db1"} Jul 16 04:46:33.965077 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:33.964973 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" Jul 16 04:46:33.985487 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:33.985437 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" podStartSLOduration=2.474596194 podStartE2EDuration="3.985425632s" totalImagesPullingTime="1.510829438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:46:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:46:33.98410505 +0000 UTC m=+446.709573583" watchObservedRunningTime="2026-07-16 04:46:33.985425632 +0000 UTC m=+446.710894153" Jul 16 04:46:34.094871 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.094840 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2f5mn"] Jul 16 04:46:34.098678 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.098658 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2f5mn" Jul 16 04:46:34.101472 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.101454 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-mj4qd\"" Jul 16 04:46:34.109625 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.109606 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2f5mn"] Jul 16 04:46:34.183754 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.183731 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhgjp\" (UniqueName: \"kubernetes.io/projected/bd850487-c720-46c5-a704-7d60238b0e6b-kube-api-access-lhgjp\") pod \"authorino-operator-75cc8d5cb7-2f5mn\" (UID: \"bd850487-c720-46c5-a704-7d60238b0e6b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2f5mn" Jul 16 04:46:34.284658 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.284601 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhgjp\" (UniqueName: \"kubernetes.io/projected/bd850487-c720-46c5-a704-7d60238b0e6b-kube-api-access-lhgjp\") pod \"authorino-operator-75cc8d5cb7-2f5mn\" (UID: \"bd850487-c720-46c5-a704-7d60238b0e6b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2f5mn" Jul 16 04:46:34.295389 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.295365 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhgjp\" (UniqueName: \"kubernetes.io/projected/bd850487-c720-46c5-a704-7d60238b0e6b-kube-api-access-lhgjp\") pod \"authorino-operator-75cc8d5cb7-2f5mn\" (UID: \"bd850487-c720-46c5-a704-7d60238b0e6b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2f5mn" Jul 16 04:46:34.409844 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.409820 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2f5mn" Jul 16 04:46:34.557225 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.557201 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2f5mn"] Jul 16 04:46:34.559688 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:46:34.559661 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd850487_c720_46c5_a704_7d60238b0e6b.slice/crio-6e14aa25b86c70c20f9f324ca90b30b44ca9f86ff744d55e2279f089db96ed61 WatchSource:0}: Error finding container 6e14aa25b86c70c20f9f324ca90b30b44ca9f86ff744d55e2279f089db96ed61: Status 404 returned error can't find the container with id 6e14aa25b86c70c20f9f324ca90b30b44ca9f86ff744d55e2279f089db96ed61 Jul 16 04:46:34.566904 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.566882 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-7qr9d"] Jul 16 04:46:34.571454 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.571437 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7qr9d" Jul 16 04:46:34.574065 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.574041 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 16 04:46:34.574268 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.574244 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:34.574352 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.574303 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8r5pn\"" Jul 16 04:46:34.574452 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.574438 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 16 04:46:34.583496 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.583475 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-7qr9d"] Jul 16 04:46:34.688843 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.688817 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c381c339-c947-4130-8568-d2eca3f56ef6-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-7qr9d\" (UID: \"c381c339-c947-4130-8568-d2eca3f56ef6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7qr9d" Jul 16 04:46:34.688965 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.688868 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c381c339-c947-4130-8568-d2eca3f56ef6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7qr9d\" (UID: \"c381c339-c947-4130-8568-d2eca3f56ef6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7qr9d" Jul 16 04:46:34.688965 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.688948 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmqh5\" (UniqueName: \"kubernetes.io/projected/c381c339-c947-4130-8568-d2eca3f56ef6-kube-api-access-hmqh5\") pod \"kuadrant-console-plugin-66b5774985-7qr9d\" (UID: \"c381c339-c947-4130-8568-d2eca3f56ef6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7qr9d" Jul 16 04:46:34.790145 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.790118 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c381c339-c947-4130-8568-d2eca3f56ef6-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-7qr9d\" (UID: \"c381c339-c947-4130-8568-d2eca3f56ef6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7qr9d" Jul 16 04:46:34.790281 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.790168 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c381c339-c947-4130-8568-d2eca3f56ef6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7qr9d\" (UID: \"c381c339-c947-4130-8568-d2eca3f56ef6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7qr9d" Jul 16 04:46:34.790281 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.790192 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmqh5\" (UniqueName: \"kubernetes.io/projected/c381c339-c947-4130-8568-d2eca3f56ef6-kube-api-access-hmqh5\") pod \"kuadrant-console-plugin-66b5774985-7qr9d\" (UID: \"c381c339-c947-4130-8568-d2eca3f56ef6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7qr9d" Jul 16 04:46:34.790395 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:34.790327 2581 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 16 04:46:34.790436 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:34.790404 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c381c339-c947-4130-8568-d2eca3f56ef6-plugin-serving-cert podName:c381c339-c947-4130-8568-d2eca3f56ef6 nodeName:}" failed. No retries permitted until 2026-07-16 04:46:35.290381839 +0000 UTC m=+448.015850342 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/c381c339-c947-4130-8568-d2eca3f56ef6-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-7qr9d" (UID: "c381c339-c947-4130-8568-d2eca3f56ef6") : secret "plugin-serving-cert" not found Jul 16 04:46:34.790865 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.790848 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c381c339-c947-4130-8568-d2eca3f56ef6-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-7qr9d\" (UID: \"c381c339-c947-4130-8568-d2eca3f56ef6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7qr9d" Jul 16 04:46:34.799768 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.799746 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmqh5\" (UniqueName: \"kubernetes.io/projected/c381c339-c947-4130-8568-d2eca3f56ef6-kube-api-access-hmqh5\") pod \"kuadrant-console-plugin-66b5774985-7qr9d\" (UID: \"c381c339-c947-4130-8568-d2eca3f56ef6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7qr9d" Jul 16 04:46:34.969948 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.969918 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2f5mn" event={"ID":"bd850487-c720-46c5-a704-7d60238b0e6b","Type":"ContainerStarted","Data":"6e14aa25b86c70c20f9f324ca90b30b44ca9f86ff744d55e2279f089db96ed61"} Jul 16 04:46:34.974776 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:34.974751 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:35.294602 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:35.294530 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c381c339-c947-4130-8568-d2eca3f56ef6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7qr9d\" (UID: \"c381c339-c947-4130-8568-d2eca3f56ef6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7qr9d" Jul 16 04:46:35.297147 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:35.297095 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c381c339-c947-4130-8568-d2eca3f56ef6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7qr9d\" (UID: \"c381c339-c947-4130-8568-d2eca3f56ef6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7qr9d" Jul 16 04:46:35.482623 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:35.482588 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7qr9d" Jul 16 04:46:35.636055 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:35.636023 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-7qr9d"] Jul 16 04:46:35.638728 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:46:35.638702 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc381c339_c947_4130_8568_d2eca3f56ef6.slice/crio-26378412efc1de686e91b5d1c03d1abe19e39e006315632336c0a4461e06b176 WatchSource:0}: Error finding container 26378412efc1de686e91b5d1c03d1abe19e39e006315632336c0a4461e06b176: Status 404 returned error can't find the container with id 26378412efc1de686e91b5d1c03d1abe19e39e006315632336c0a4461e06b176 Jul 16 04:46:35.977180 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:35.977142 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7qr9d" event={"ID":"c381c339-c947-4130-8568-d2eca3f56ef6","Type":"ContainerStarted","Data":"26378412efc1de686e91b5d1c03d1abe19e39e006315632336c0a4461e06b176"} Jul 16 04:46:36.090388 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:36.090351 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:36.190886 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:36.190835 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:36.985579 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:36.985528 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2f5mn" event={"ID":"bd850487-c720-46c5-a704-7d60238b0e6b","Type":"ContainerStarted","Data":"ee9f7dfd6e302fd2ed8c33be894287f4fb08e9264b161a787fd5f9efa3d4fd0c"} Jul 16 04:46:36.986154 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:36.985729 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2f5mn" Jul 16 04:46:37.006011 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:37.005960 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2f5mn" podStartSLOduration=0.884093065 podStartE2EDuration="3.005940621s" totalImagesPullingTime="2.121847556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:46:37.003844497 +0000 UTC m=+449.729313022" watchObservedRunningTime="2026-07-16 04:46:37.005940621 +0000 UTC m=+449.731409142" Jul 16 04:46:38.191353 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:38.191312 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:38.290441 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:38.290404 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:42.961466 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:42.961434 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:42.961934 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:42.961804 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s8f5d" Jul 16 04:46:44.786946 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.786912 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94"] Jul 16 04:46:44.787395 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.787151 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" podUID="3db2226b-912a-4512-967f-8a2c97793ebc" containerName="manager" containerID="cri-o://9ec4f0df53e38c9dd13a87602fd317944e0455ded6cf6b31d7d25fc48f156ba7" gracePeriod=2 Jul 16 04:46:44.792114 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.791304 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94"] Jul 16 04:46:44.792114 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:44.791537 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-k5t94\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" podUID="3db2226b-912a-4512-967f-8a2c97793ebc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:44.802950 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.802926 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw"] Jul 16 04:46:44.803187 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.803157 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" podUID="87a513d6-0edb-467e-831f-6f1673ae9f42" containerName="manager" containerID="cri-o://904a71d26fb5caf04df44795db1bd3ee076d08eeb127591143e367c7c6a97db1" gracePeriod=2 Jul 16 04:46:44.805257 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.805236 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" Jul 16 04:46:44.806841 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:44.806799 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-k5t94\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" podUID="3db2226b-912a-4512-967f-8a2c97793ebc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:44.810090 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:44.810032 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rmtkw\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" podUID="87a513d6-0edb-467e-831f-6f1673ae9f42" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" Jul 16 04:46:44.811713 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.811651 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw"] Jul 16 04:46:44.814034 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:44.813996 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-k5t94\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" podUID="3db2226b-912a-4512-967f-8a2c97793ebc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:44.818226 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:44.818182 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rmtkw\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" podUID="87a513d6-0edb-467e-831f-6f1673ae9f42" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" Jul 16 04:46:44.822269 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.822252 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t"] Jul 16 04:46:44.822680 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.822663 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87a513d6-0edb-467e-831f-6f1673ae9f42" containerName="manager" Jul 16 04:46:44.822770 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.822693 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="87a513d6-0edb-467e-831f-6f1673ae9f42" containerName="manager" Jul 16 04:46:44.822770 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.822714 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3db2226b-912a-4512-967f-8a2c97793ebc" containerName="manager" Jul 16 04:46:44.822770 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.822723 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3db2226b-912a-4512-967f-8a2c97793ebc" containerName="manager" Jul 16 04:46:44.822910 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.822842 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3db2226b-912a-4512-967f-8a2c97793ebc" containerName="manager" Jul 16 04:46:44.822910 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.822855 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="87a513d6-0edb-467e-831f-6f1673ae9f42" containerName="manager" Jul 16 04:46:44.827817 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.827785 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t" Jul 16 04:46:44.830651 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.830631 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8f88k"] Jul 16 04:46:44.835927 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.835896 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8f88k" Jul 16 04:46:44.836449 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.836430 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t"] Jul 16 04:46:44.849889 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.849802 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8f88k"] Jul 16 04:46:44.856345 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:44.856314 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-k5t94\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" podUID="3db2226b-912a-4512-967f-8a2c97793ebc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:44.858631 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:44.858601 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rmtkw\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" podUID="87a513d6-0edb-467e-831f-6f1673ae9f42" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" Jul 16 04:46:44.860862 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:44.860832 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rmtkw\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" podUID="87a513d6-0edb-467e-831f-6f1673ae9f42" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" Jul 16 04:46:44.879758 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.879729 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05d138ca-938f-4e69-9cb9-b90a8a3d5b9b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j4n4t\" (UID: \"05d138ca-938f-4e69-9cb9-b90a8a3d5b9b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t" Jul 16 04:46:44.879906 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.879885 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw7dp\" (UniqueName: \"kubernetes.io/projected/05d138ca-938f-4e69-9cb9-b90a8a3d5b9b-kube-api-access-fw7dp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j4n4t\" (UID: \"05d138ca-938f-4e69-9cb9-b90a8a3d5b9b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t" Jul 16 04:46:44.887971 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:44.887945 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-k5t94\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" podUID="3db2226b-912a-4512-967f-8a2c97793ebc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:44.980787 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.980755 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05d138ca-938f-4e69-9cb9-b90a8a3d5b9b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j4n4t\" (UID: \"05d138ca-938f-4e69-9cb9-b90a8a3d5b9b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t" Jul 16 04:46:44.980928 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.980860 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj5m5\" (UniqueName: \"kubernetes.io/projected/d66f64fc-f7c0-4b56-befd-6fad4182d62e-kube-api-access-wj5m5\") pod \"limitador-operator-controller-manager-859d7b78bd-8f88k\" (UID: \"d66f64fc-f7c0-4b56-befd-6fad4182d62e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8f88k" Jul 16 04:46:44.980928 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.980890 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw7dp\" (UniqueName: \"kubernetes.io/projected/05d138ca-938f-4e69-9cb9-b90a8a3d5b9b-kube-api-access-fw7dp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j4n4t\" (UID: \"05d138ca-938f-4e69-9cb9-b90a8a3d5b9b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t" Jul 16 04:46:44.981228 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.981159 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05d138ca-938f-4e69-9cb9-b90a8a3d5b9b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j4n4t\" (UID: \"05d138ca-938f-4e69-9cb9-b90a8a3d5b9b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t" Jul 16 04:46:44.993900 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:44.993878 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw7dp\" (UniqueName: \"kubernetes.io/projected/05d138ca-938f-4e69-9cb9-b90a8a3d5b9b-kube-api-access-fw7dp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j4n4t\" (UID: \"05d138ca-938f-4e69-9cb9-b90a8a3d5b9b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t" Jul 16 04:46:45.051793 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:45.051720 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:45.081855 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:45.081829 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj5m5\" (UniqueName: \"kubernetes.io/projected/d66f64fc-f7c0-4b56-befd-6fad4182d62e-kube-api-access-wj5m5\") pod \"limitador-operator-controller-manager-859d7b78bd-8f88k\" (UID: \"d66f64fc-f7c0-4b56-befd-6fad4182d62e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8f88k" Jul 16 04:46:45.094159 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:45.094109 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj5m5\" (UniqueName: \"kubernetes.io/projected/d66f64fc-f7c0-4b56-befd-6fad4182d62e-kube-api-access-wj5m5\") pod \"limitador-operator-controller-manager-859d7b78bd-8f88k\" (UID: \"d66f64fc-f7c0-4b56-befd-6fad4182d62e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8f88k" Jul 16 04:46:45.149674 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:45.149626 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:45.232141 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:45.232112 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t" Jul 16 04:46:45.239101 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:45.239076 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8f88k" Jul 16 04:46:47.995799 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:47.995764 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2f5mn" Jul 16 04:46:49.275810 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:49.275779 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:49.382292 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:49.382168 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:53.812631 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:53.812586 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:53.874972 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:53.874925 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:54.462754 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:54.462711 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:54.564953 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:54.564917 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:46:57.626111 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:57.623734 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8f88k"] Jul 16 04:46:57.653698 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:57.653668 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t"] Jul 16 04:46:57.717353 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:46:57.717314 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66f64fc_f7c0_4b56_befd_6fad4182d62e.slice/crio-68601d1258bf422159f101ee9c4c9fb77f5f8d1c1eee7df64f268e65ad837ea4 WatchSource:0}: Error finding container 68601d1258bf422159f101ee9c4c9fb77f5f8d1c1eee7df64f268e65ad837ea4: Status 404 returned error can't find the container with id 68601d1258bf422159f101ee9c4c9fb77f5f8d1c1eee7df64f268e65ad837ea4 Jul 16 04:46:57.718008 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:46:57.717981 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05d138ca_938f_4e69_9cb9_b90a8a3d5b9b.slice/crio-3f24bde343581846c1ea3dac58c5c6e7239c533d738ad3fa2a6937abe8a2f71c WatchSource:0}: Error finding container 3f24bde343581846c1ea3dac58c5c6e7239c533d738ad3fa2a6937abe8a2f71c: Status 404 returned error can't find the container with id 3f24bde343581846c1ea3dac58c5c6e7239c533d738ad3fa2a6937abe8a2f71c Jul 16 04:46:57.741611 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:57.741588 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" Jul 16 04:46:57.743932 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:57.743907 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rmtkw\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" podUID="87a513d6-0edb-467e-831f-6f1673ae9f42" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" Jul 16 04:46:57.749992 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:57.749935 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:57.752314 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:57.752282 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rmtkw\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" podUID="87a513d6-0edb-467e-831f-6f1673ae9f42" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" Jul 16 04:46:57.754378 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:57.754345 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-k5t94\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" podUID="3db2226b-912a-4512-967f-8a2c97793ebc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:57.844275 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:57.841318 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-k5t94\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" podUID="3db2226b-912a-4512-967f-8a2c97793ebc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:57.844275 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:57.844216 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rmtkw\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" podUID="87a513d6-0edb-467e-831f-6f1673ae9f42" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" Jul 16 04:46:57.902354 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:57.902329 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwr9v\" (UniqueName: \"kubernetes.io/projected/87a513d6-0edb-467e-831f-6f1673ae9f42-kube-api-access-vwr9v\") pod \"87a513d6-0edb-467e-831f-6f1673ae9f42\" (UID: \"87a513d6-0edb-467e-831f-6f1673ae9f42\") " Jul 16 04:46:57.902556 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:57.902374 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnrrh\" (UniqueName: \"kubernetes.io/projected/3db2226b-912a-4512-967f-8a2c97793ebc-kube-api-access-tnrrh\") pod \"3db2226b-912a-4512-967f-8a2c97793ebc\" (UID: \"3db2226b-912a-4512-967f-8a2c97793ebc\") " Jul 16 04:46:57.902556 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:57.902399 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3db2226b-912a-4512-967f-8a2c97793ebc-extensions-socket-volume\") pod \"3db2226b-912a-4512-967f-8a2c97793ebc\" (UID: \"3db2226b-912a-4512-967f-8a2c97793ebc\") " Jul 16 04:46:57.902959 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:57.902933 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3db2226b-912a-4512-967f-8a2c97793ebc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3db2226b-912a-4512-967f-8a2c97793ebc" (UID: "3db2226b-912a-4512-967f-8a2c97793ebc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:46:57.904276 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:57.904253 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3db2226b-912a-4512-967f-8a2c97793ebc-kube-api-access-tnrrh" (OuterVolumeSpecName: "kube-api-access-tnrrh") pod "3db2226b-912a-4512-967f-8a2c97793ebc" (UID: "3db2226b-912a-4512-967f-8a2c97793ebc"). InnerVolumeSpecName "kube-api-access-tnrrh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:46:57.904416 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:57.904399 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87a513d6-0edb-467e-831f-6f1673ae9f42-kube-api-access-vwr9v" (OuterVolumeSpecName: "kube-api-access-vwr9v") pod "87a513d6-0edb-467e-831f-6f1673ae9f42" (UID: "87a513d6-0edb-467e-831f-6f1673ae9f42"). InnerVolumeSpecName "kube-api-access-vwr9v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:46:58.003913 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.003832 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwr9v\" (UniqueName: \"kubernetes.io/projected/87a513d6-0edb-467e-831f-6f1673ae9f42-kube-api-access-vwr9v\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:46:58.003913 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.003859 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnrrh\" (UniqueName: \"kubernetes.io/projected/3db2226b-912a-4512-967f-8a2c97793ebc-kube-api-access-tnrrh\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:46:58.003913 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.003871 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3db2226b-912a-4512-967f-8a2c97793ebc-extensions-socket-volume\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:46:58.104641 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.104607 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8f88k" event={"ID":"d66f64fc-f7c0-4b56-befd-6fad4182d62e","Type":"ContainerStarted","Data":"501c207075a521f3bbe2f1ccb8992f27383774424ac709d2e71c35dd981bac8b"} Jul 16 04:46:58.104818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.104665 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8f88k" event={"ID":"d66f64fc-f7c0-4b56-befd-6fad4182d62e","Type":"ContainerStarted","Data":"68601d1258bf422159f101ee9c4c9fb77f5f8d1c1eee7df64f268e65ad837ea4"} Jul 16 04:46:58.104818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.104703 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8f88k" Jul 16 04:46:58.106178 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.106151 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t" event={"ID":"05d138ca-938f-4e69-9cb9-b90a8a3d5b9b","Type":"ContainerStarted","Data":"3d50bddace3ee60c20ddd52efdada7806a9a817f228cb8f4ab027653557cf4a8"} Jul 16 04:46:58.106299 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.106209 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t" event={"ID":"05d138ca-938f-4e69-9cb9-b90a8a3d5b9b","Type":"ContainerStarted","Data":"3f24bde343581846c1ea3dac58c5c6e7239c533d738ad3fa2a6937abe8a2f71c"} Jul 16 04:46:58.106299 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.106232 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t" Jul 16 04:46:58.107284 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.107265 2581 generic.go:356] "Generic (PLEG): container finished" podID="3db2226b-912a-4512-967f-8a2c97793ebc" containerID="9ec4f0df53e38c9dd13a87602fd317944e0455ded6cf6b31d7d25fc48f156ba7" exitCode=0 Jul 16 04:46:58.107376 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.107309 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:58.107376 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.107344 2581 scope.go:117] "RemoveContainer" containerID="9ec4f0df53e38c9dd13a87602fd317944e0455ded6cf6b31d7d25fc48f156ba7" Jul 16 04:46:58.108654 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.108629 2581 generic.go:356] "Generic (PLEG): container finished" podID="87a513d6-0edb-467e-831f-6f1673ae9f42" containerID="904a71d26fb5caf04df44795db1bd3ee076d08eeb127591143e367c7c6a97db1" exitCode=0 Jul 16 04:46:58.108736 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.108681 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" Jul 16 04:46:58.110262 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.110239 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7qr9d" event={"ID":"c381c339-c947-4130-8568-d2eca3f56ef6","Type":"ContainerStarted","Data":"a713bec1f92330955b0a5612744b89faa0b4c6f311962888c9592cad78f8f5aa"} Jul 16 04:46:58.116032 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.116012 2581 scope.go:117] "RemoveContainer" containerID="9ec4f0df53e38c9dd13a87602fd317944e0455ded6cf6b31d7d25fc48f156ba7" Jul 16 04:46:58.116289 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:58.116271 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ec4f0df53e38c9dd13a87602fd317944e0455ded6cf6b31d7d25fc48f156ba7\": container with ID starting with 9ec4f0df53e38c9dd13a87602fd317944e0455ded6cf6b31d7d25fc48f156ba7 not found: ID does not exist" containerID="9ec4f0df53e38c9dd13a87602fd317944e0455ded6cf6b31d7d25fc48f156ba7" Jul 16 04:46:58.116349 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.116297 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ec4f0df53e38c9dd13a87602fd317944e0455ded6cf6b31d7d25fc48f156ba7"} err="failed to get container status \"9ec4f0df53e38c9dd13a87602fd317944e0455ded6cf6b31d7d25fc48f156ba7\": rpc error: code = NotFound desc = could not find container \"9ec4f0df53e38c9dd13a87602fd317944e0455ded6cf6b31d7d25fc48f156ba7\": container with ID starting with 9ec4f0df53e38c9dd13a87602fd317944e0455ded6cf6b31d7d25fc48f156ba7 not found: ID does not exist" Jul 16 04:46:58.116349 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.116316 2581 scope.go:117] "RemoveContainer" containerID="904a71d26fb5caf04df44795db1bd3ee076d08eeb127591143e367c7c6a97db1" Jul 16 04:46:58.124927 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.124864 2581 scope.go:117] "RemoveContainer" containerID="904a71d26fb5caf04df44795db1bd3ee076d08eeb127591143e367c7c6a97db1" Jul 16 04:46:58.124991 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.124841 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8f88k" podStartSLOduration=14.124822953 podStartE2EDuration="14.124822953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:46:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:46:58.12448449 +0000 UTC m=+470.849953032" watchObservedRunningTime="2026-07-16 04:46:58.124822953 +0000 UTC m=+470.850291478" Jul 16 04:46:58.125170 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:58.125151 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"904a71d26fb5caf04df44795db1bd3ee076d08eeb127591143e367c7c6a97db1\": container with ID starting with 904a71d26fb5caf04df44795db1bd3ee076d08eeb127591143e367c7c6a97db1 not found: ID does not exist" containerID="904a71d26fb5caf04df44795db1bd3ee076d08eeb127591143e367c7c6a97db1" Jul 16 04:46:58.125222 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.125180 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"904a71d26fb5caf04df44795db1bd3ee076d08eeb127591143e367c7c6a97db1"} err="failed to get container status \"904a71d26fb5caf04df44795db1bd3ee076d08eeb127591143e367c7c6a97db1\": rpc error: code = NotFound desc = could not find container \"904a71d26fb5caf04df44795db1bd3ee076d08eeb127591143e367c7c6a97db1\": container with ID starting with 904a71d26fb5caf04df44795db1bd3ee076d08eeb127591143e367c7c6a97db1 not found: ID does not exist" Jul 16 04:46:58.142844 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.142797 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t" podStartSLOduration=14.142780104 podStartE2EDuration="14.142780104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:46:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:46:58.142289048 +0000 UTC m=+470.867757594" watchObservedRunningTime="2026-07-16 04:46:58.142780104 +0000 UTC m=+470.868248619" Jul 16 04:46:58.144660 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:58.144630 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-k5t94\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" podUID="3db2226b-912a-4512-967f-8a2c97793ebc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k5t94" Jul 16 04:46:58.166489 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:58.166440 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7qr9d" podStartSLOduration=2.038295527 podStartE2EDuration="24.166424523s" totalImagesPullingTime="22.128128996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:46:58.165574939 +0000 UTC m=+470.891043487" watchObservedRunningTime="2026-07-16 04:46:58.166424523 +0000 UTC m=+470.891893054" Jul 16 04:46:58.167957 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:46:58.167922 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rmtkw\" is forbidden: User \"system:node:ip-10-0-132-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-245.ec2.internal' and this object" podUID="87a513d6-0edb-467e-831f-6f1673ae9f42" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rmtkw" Jul 16 04:46:59.838411 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:59.838375 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3db2226b-912a-4512-967f-8a2c97793ebc" path="/var/lib/kubelet/pods/3db2226b-912a-4512-967f-8a2c97793ebc/volumes" Jul 16 04:46:59.838896 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:46:59.838865 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87a513d6-0edb-467e-831f-6f1673ae9f42" path="/var/lib/kubelet/pods/87a513d6-0edb-467e-831f-6f1673ae9f42/volumes" Jul 16 04:47:03.790988 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:03.790938 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t"] Jul 16 04:47:03.791646 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:03.791194 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t" podUID="05d138ca-938f-4e69-9cb9-b90a8a3d5b9b" containerName="manager" containerID="cri-o://3d50bddace3ee60c20ddd52efdada7806a9a817f228cb8f4ab027653557cf4a8" gracePeriod=10 Jul 16 04:47:03.793475 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:03.793449 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t" Jul 16 04:47:04.142730 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:04.142658 2581 generic.go:356] "Generic (PLEG): container finished" podID="05d138ca-938f-4e69-9cb9-b90a8a3d5b9b" containerID="3d50bddace3ee60c20ddd52efdada7806a9a817f228cb8f4ab027653557cf4a8" exitCode=0 Jul 16 04:47:04.142730 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:04.142710 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t" event={"ID":"05d138ca-938f-4e69-9cb9-b90a8a3d5b9b","Type":"ContainerDied","Data":"3d50bddace3ee60c20ddd52efdada7806a9a817f228cb8f4ab027653557cf4a8"} Jul 16 04:47:04.613241 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:04.613207 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t" Jul 16 04:47:04.765715 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:04.765682 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05d138ca-938f-4e69-9cb9-b90a8a3d5b9b-extensions-socket-volume\") pod \"05d138ca-938f-4e69-9cb9-b90a8a3d5b9b\" (UID: \"05d138ca-938f-4e69-9cb9-b90a8a3d5b9b\") " Jul 16 04:47:04.765889 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:04.765726 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fw7dp\" (UniqueName: \"kubernetes.io/projected/05d138ca-938f-4e69-9cb9-b90a8a3d5b9b-kube-api-access-fw7dp\") pod \"05d138ca-938f-4e69-9cb9-b90a8a3d5b9b\" (UID: \"05d138ca-938f-4e69-9cb9-b90a8a3d5b9b\") " Jul 16 04:47:04.766064 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:04.766041 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05d138ca-938f-4e69-9cb9-b90a8a3d5b9b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "05d138ca-938f-4e69-9cb9-b90a8a3d5b9b" (UID: "05d138ca-938f-4e69-9cb9-b90a8a3d5b9b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:47:04.767799 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:04.767775 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05d138ca-938f-4e69-9cb9-b90a8a3d5b9b-kube-api-access-fw7dp" (OuterVolumeSpecName: "kube-api-access-fw7dp") pod "05d138ca-938f-4e69-9cb9-b90a8a3d5b9b" (UID: "05d138ca-938f-4e69-9cb9-b90a8a3d5b9b"). InnerVolumeSpecName "kube-api-access-fw7dp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:47:04.866392 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:04.866353 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05d138ca-938f-4e69-9cb9-b90a8a3d5b9b-extensions-socket-volume\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:47:04.866392 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:04.866385 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fw7dp\" (UniqueName: \"kubernetes.io/projected/05d138ca-938f-4e69-9cb9-b90a8a3d5b9b-kube-api-access-fw7dp\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:47:05.148837 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:05.148763 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t" Jul 16 04:47:05.148837 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:05.148788 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t" event={"ID":"05d138ca-938f-4e69-9cb9-b90a8a3d5b9b","Type":"ContainerDied","Data":"3f24bde343581846c1ea3dac58c5c6e7239c533d738ad3fa2a6937abe8a2f71c"} Jul 16 04:47:05.149057 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:05.148851 2581 scope.go:117] "RemoveContainer" containerID="3d50bddace3ee60c20ddd52efdada7806a9a817f228cb8f4ab027653557cf4a8" Jul 16 04:47:05.178689 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:05.178657 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t"] Jul 16 04:47:05.182825 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:05.182795 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4n4t"] Jul 16 04:47:05.838920 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:05.838884 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05d138ca-938f-4e69-9cb9-b90a8a3d5b9b" path="/var/lib/kubelet/pods/05d138ca-938f-4e69-9cb9-b90a8a3d5b9b/volumes" Jul 16 04:47:09.119668 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:09.119633 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8f88k" Jul 16 04:47:20.232476 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.232442 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g"] Jul 16 04:47:20.232886 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.232856 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05d138ca-938f-4e69-9cb9-b90a8a3d5b9b" containerName="manager" Jul 16 04:47:20.232886 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.232870 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="05d138ca-938f-4e69-9cb9-b90a8a3d5b9b" containerName="manager" Jul 16 04:47:20.232966 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.232932 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="05d138ca-938f-4e69-9cb9-b90a8a3d5b9b" containerName="manager" Jul 16 04:47:20.284452 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.284422 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g"] Jul 16 04:47:20.284629 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.284561 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.287556 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.287532 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-8vdnq\"" Jul 16 04:47:20.296822 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.296791 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:47:20.389853 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.389817 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c241788-0107-4d03-a926-7a4bf3c806db-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.390024 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.389870 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c241788-0107-4d03-a926-7a4bf3c806db-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.390024 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.389977 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c241788-0107-4d03-a926-7a4bf3c806db-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.390024 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.390019 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c241788-0107-4d03-a926-7a4bf3c806db-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.390163 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.390040 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c241788-0107-4d03-a926-7a4bf3c806db-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.390163 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.390057 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c241788-0107-4d03-a926-7a4bf3c806db-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.390163 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.390097 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c241788-0107-4d03-a926-7a4bf3c806db-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.390163 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.390128 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c241788-0107-4d03-a926-7a4bf3c806db-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.390163 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.390157 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7w54\" (UniqueName: \"kubernetes.io/projected/9c241788-0107-4d03-a926-7a4bf3c806db-kube-api-access-c7w54\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.485498 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.485406 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:47:20.490988 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.490964 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c241788-0107-4d03-a926-7a4bf3c806db-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.491082 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.491010 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c241788-0107-4d03-a926-7a4bf3c806db-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.491082 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.491040 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c241788-0107-4d03-a926-7a4bf3c806db-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.491082 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.491060 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c241788-0107-4d03-a926-7a4bf3c806db-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.491207 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.491083 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c241788-0107-4d03-a926-7a4bf3c806db-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.491207 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.491099 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c241788-0107-4d03-a926-7a4bf3c806db-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.491207 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.491180 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c241788-0107-4d03-a926-7a4bf3c806db-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.491351 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.491244 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c241788-0107-4d03-a926-7a4bf3c806db-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.491351 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.491271 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7w54\" (UniqueName: \"kubernetes.io/projected/9c241788-0107-4d03-a926-7a4bf3c806db-kube-api-access-c7w54\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.491554 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.491526 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c241788-0107-4d03-a926-7a4bf3c806db-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.491702 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.491620 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c241788-0107-4d03-a926-7a4bf3c806db-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.491702 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.491674 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c241788-0107-4d03-a926-7a4bf3c806db-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.492045 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.492020 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c241788-0107-4d03-a926-7a4bf3c806db-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.492086 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.492032 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c241788-0107-4d03-a926-7a4bf3c806db-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.493372 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.493344 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c241788-0107-4d03-a926-7a4bf3c806db-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.493603 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.493587 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c241788-0107-4d03-a926-7a4bf3c806db-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.500948 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.500926 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c241788-0107-4d03-a926-7a4bf3c806db-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.501070 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.501047 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7w54\" (UniqueName: \"kubernetes.io/projected/9c241788-0107-4d03-a926-7a4bf3c806db-kube-api-access-c7w54\") pod \"maas-default-gateway-openshift-default-56654dcf7-llv9g\" (UID: \"9c241788-0107-4d03-a926-7a4bf3c806db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.594651 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.594619 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:20.729896 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.729874 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g"] Jul 16 04:47:20.731706 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:47:20.731676 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c241788_0107_4d03_a926_7a4bf3c806db.slice/crio-028508ad5c2b8bc5444a5eb0ba91842fbff32bc47d1025fa24f8e9e890f75ccc WatchSource:0}: Error finding container 028508ad5c2b8bc5444a5eb0ba91842fbff32bc47d1025fa24f8e9e890f75ccc: Status 404 returned error can't find the container with id 028508ad5c2b8bc5444a5eb0ba91842fbff32bc47d1025fa24f8e9e890f75ccc Jul 16 04:47:20.734020 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.733991 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 04:47:20.734112 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.734052 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 04:47:20.734112 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:20.734085 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 04:47:21.228219 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:21.228187 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" event={"ID":"9c241788-0107-4d03-a926-7a4bf3c806db","Type":"ContainerStarted","Data":"504f085deb0f6236c46784519f30efcc09168aa051b98cec5919e688baadff8f"} Jul 16 04:47:21.228385 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:21.228235 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" event={"ID":"9c241788-0107-4d03-a926-7a4bf3c806db","Type":"ContainerStarted","Data":"028508ad5c2b8bc5444a5eb0ba91842fbff32bc47d1025fa24f8e9e890f75ccc"} Jul 16 04:47:21.251672 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:21.251619 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" podStartSLOduration=1.251601945 podStartE2EDuration="1.251601945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:47:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:47:21.249945769 +0000 UTC m=+493.975414349" watchObservedRunningTime="2026-07-16 04:47:21.251601945 +0000 UTC m=+493.977070468" Jul 16 04:47:21.595320 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:21.595240 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:21.599560 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:21.599533 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:22.233580 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:22.233550 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:22.234370 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:22.234354 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:47:22.338525 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:22.338477 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jkv82"] Jul 16 04:47:22.339236 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:22.339208 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:47:22.340905 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:22.340884 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkv82" Jul 16 04:47:22.344833 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:22.344817 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 16 04:47:22.355242 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:22.355222 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jkv82"] Jul 16 04:47:22.409567 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:22.409470 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7fd2\" (UniqueName: \"kubernetes.io/projected/b463b567-fa2d-4221-bed0-01e505b97504-kube-api-access-d7fd2\") pod \"limitador-limitador-64c94fd64f-jkv82\" (UID: \"b463b567-fa2d-4221-bed0-01e505b97504\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jkv82" Jul 16 04:47:22.409793 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:22.409681 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b463b567-fa2d-4221-bed0-01e505b97504-config-file\") pod \"limitador-limitador-64c94fd64f-jkv82\" (UID: \"b463b567-fa2d-4221-bed0-01e505b97504\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jkv82" Jul 16 04:47:22.426003 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:22.425977 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jkv82"] Jul 16 04:47:22.428728 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:22.428698 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:47:22.495322 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:22.495244 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:47:22.510373 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:22.510351 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b463b567-fa2d-4221-bed0-01e505b97504-config-file\") pod \"limitador-limitador-64c94fd64f-jkv82\" (UID: \"b463b567-fa2d-4221-bed0-01e505b97504\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jkv82" Jul 16 04:47:22.510451 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:22.510394 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7fd2\" (UniqueName: \"kubernetes.io/projected/b463b567-fa2d-4221-bed0-01e505b97504-kube-api-access-d7fd2\") pod \"limitador-limitador-64c94fd64f-jkv82\" (UID: \"b463b567-fa2d-4221-bed0-01e505b97504\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jkv82" Jul 16 04:47:22.511091 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:22.511070 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b463b567-fa2d-4221-bed0-01e505b97504-config-file\") pod \"limitador-limitador-64c94fd64f-jkv82\" (UID: \"b463b567-fa2d-4221-bed0-01e505b97504\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jkv82" Jul 16 04:47:22.520188 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:22.520154 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7fd2\" (UniqueName: \"kubernetes.io/projected/b463b567-fa2d-4221-bed0-01e505b97504-kube-api-access-d7fd2\") pod \"limitador-limitador-64c94fd64f-jkv82\" (UID: \"b463b567-fa2d-4221-bed0-01e505b97504\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jkv82" Jul 16 04:47:22.651263 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:22.651228 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkv82" Jul 16 04:47:22.780346 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:22.780324 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jkv82"] Jul 16 04:47:22.782321 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:47:22.782293 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb463b567_fa2d_4221_bed0_01e505b97504.slice/crio-c20a4cd1dc2879b240ae2386ecd340cbfb95a879ba72eb0eacebe329c35af097 WatchSource:0}: Error finding container c20a4cd1dc2879b240ae2386ecd340cbfb95a879ba72eb0eacebe329c35af097: Status 404 returned error can't find the container with id c20a4cd1dc2879b240ae2386ecd340cbfb95a879ba72eb0eacebe329c35af097 Jul 16 04:47:23.155357 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:23.155265 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:47:23.241822 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:23.241789 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkv82" event={"ID":"b463b567-fa2d-4221-bed0-01e505b97504","Type":"ContainerStarted","Data":"c20a4cd1dc2879b240ae2386ecd340cbfb95a879ba72eb0eacebe329c35af097"} Jul 16 04:47:23.257075 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:23.257045 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:47:23.331627 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:23.331603 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-wwzng"] Jul 16 04:47:23.333739 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:23.333717 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wwzng" Jul 16 04:47:23.336130 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:23.336110 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2m5jz\"" Jul 16 04:47:23.342999 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:23.342975 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wwzng"] Jul 16 04:47:23.419303 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:23.419225 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl4hs\" (UniqueName: \"kubernetes.io/projected/742209b3-9108-40b4-b2cf-fb2b5f30510c-kube-api-access-bl4hs\") pod \"authorino-75d556bd77-wwzng\" (UID: \"742209b3-9108-40b4-b2cf-fb2b5f30510c\") " pod="kuadrant-system/authorino-75d556bd77-wwzng" Jul 16 04:47:23.519828 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:23.519786 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl4hs\" (UniqueName: \"kubernetes.io/projected/742209b3-9108-40b4-b2cf-fb2b5f30510c-kube-api-access-bl4hs\") pod \"authorino-75d556bd77-wwzng\" (UID: \"742209b3-9108-40b4-b2cf-fb2b5f30510c\") " pod="kuadrant-system/authorino-75d556bd77-wwzng" Jul 16 04:47:23.531033 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:23.530941 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl4hs\" (UniqueName: \"kubernetes.io/projected/742209b3-9108-40b4-b2cf-fb2b5f30510c-kube-api-access-bl4hs\") pod \"authorino-75d556bd77-wwzng\" (UID: \"742209b3-9108-40b4-b2cf-fb2b5f30510c\") " pod="kuadrant-system/authorino-75d556bd77-wwzng" Jul 16 04:47:23.644193 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:23.644157 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wwzng" Jul 16 04:47:23.812886 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:23.812859 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wwzng"] Jul 16 04:47:23.814899 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:47:23.814869 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod742209b3_9108_40b4_b2cf_fb2b5f30510c.slice/crio-a84c8091d11648c16509fffbddbefd20c7ae22695908a677cceaec53e3d52e72 WatchSource:0}: Error finding container a84c8091d11648c16509fffbddbefd20c7ae22695908a677cceaec53e3d52e72: Status 404 returned error can't find the container with id a84c8091d11648c16509fffbddbefd20c7ae22695908a677cceaec53e3d52e72 Jul 16 04:47:24.250296 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:24.250260 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wwzng" event={"ID":"742209b3-9108-40b4-b2cf-fb2b5f30510c","Type":"ContainerStarted","Data":"a84c8091d11648c16509fffbddbefd20c7ae22695908a677cceaec53e3d52e72"} Jul 16 04:47:26.263069 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:26.263038 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkv82" event={"ID":"b463b567-fa2d-4221-bed0-01e505b97504","Type":"ContainerStarted","Data":"3737631eab03aa87327b380a83d5220343cc7a453c811925d8795d38c1905c51"} Jul 16 04:47:26.263448 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:26.263163 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkv82" Jul 16 04:47:26.281111 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:26.281065 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkv82" podStartSLOduration=0.958460015 podStartE2EDuration="4.28104781s" totalImagesPullingTime="3.322587795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:47:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:47:26.279307303 +0000 UTC m=+499.004775880" watchObservedRunningTime="2026-07-16 04:47:26.28104781 +0000 UTC m=+499.006516379" Jul 16 04:47:28.274127 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:28.274095 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wwzng" event={"ID":"742209b3-9108-40b4-b2cf-fb2b5f30510c","Type":"ContainerStarted","Data":"f39b38665e79db027b7d663d2ce447c2a6ff32769110d09746add1cdaeeb4f93"} Jul 16 04:47:28.290108 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:28.290060 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-wwzng" podStartSLOduration=1.055145764 podStartE2EDuration="5.290044821s" totalImagesPullingTime="4.234899057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:47:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:47:28.288381261 +0000 UTC m=+501.013849782" watchObservedRunningTime="2026-07-16 04:47:28.290044821 +0000 UTC m=+501.015513344" Jul 16 04:47:30.130456 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:30.130418 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:47:37.268998 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:37.268968 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkv82" Jul 16 04:47:37.385188 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:37.385155 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jkv82"] Jul 16 04:47:37.385350 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:37.385328 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkv82" podUID="b463b567-fa2d-4221-bed0-01e505b97504" containerName="limitador" containerID="cri-o://3737631eab03aa87327b380a83d5220343cc7a453c811925d8795d38c1905c51" gracePeriod=30 Jul 16 04:47:37.931160 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:37.931136 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkv82" Jul 16 04:47:38.052163 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:38.052133 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b463b567-fa2d-4221-bed0-01e505b97504-config-file\") pod \"b463b567-fa2d-4221-bed0-01e505b97504\" (UID: \"b463b567-fa2d-4221-bed0-01e505b97504\") " Jul 16 04:47:38.052328 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:38.052169 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7fd2\" (UniqueName: \"kubernetes.io/projected/b463b567-fa2d-4221-bed0-01e505b97504-kube-api-access-d7fd2\") pod \"b463b567-fa2d-4221-bed0-01e505b97504\" (UID: \"b463b567-fa2d-4221-bed0-01e505b97504\") " Jul 16 04:47:38.052554 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:38.052522 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b463b567-fa2d-4221-bed0-01e505b97504-config-file" (OuterVolumeSpecName: "config-file") pod "b463b567-fa2d-4221-bed0-01e505b97504" (UID: "b463b567-fa2d-4221-bed0-01e505b97504"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:47:38.054229 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:38.054210 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b463b567-fa2d-4221-bed0-01e505b97504-kube-api-access-d7fd2" (OuterVolumeSpecName: "kube-api-access-d7fd2") pod "b463b567-fa2d-4221-bed0-01e505b97504" (UID: "b463b567-fa2d-4221-bed0-01e505b97504"). InnerVolumeSpecName "kube-api-access-d7fd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:47:38.153666 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:38.153632 2581 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b463b567-fa2d-4221-bed0-01e505b97504-config-file\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:47:38.153666 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:38.153660 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7fd2\" (UniqueName: \"kubernetes.io/projected/b463b567-fa2d-4221-bed0-01e505b97504-kube-api-access-d7fd2\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:47:38.323974 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:38.323889 2581 generic.go:356] "Generic (PLEG): container finished" podID="b463b567-fa2d-4221-bed0-01e505b97504" containerID="3737631eab03aa87327b380a83d5220343cc7a453c811925d8795d38c1905c51" exitCode=0 Jul 16 04:47:38.323974 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:38.323934 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkv82" Jul 16 04:47:38.323974 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:38.323958 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkv82" event={"ID":"b463b567-fa2d-4221-bed0-01e505b97504","Type":"ContainerDied","Data":"3737631eab03aa87327b380a83d5220343cc7a453c811925d8795d38c1905c51"} Jul 16 04:47:38.324453 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:38.324000 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jkv82" event={"ID":"b463b567-fa2d-4221-bed0-01e505b97504","Type":"ContainerDied","Data":"c20a4cd1dc2879b240ae2386ecd340cbfb95a879ba72eb0eacebe329c35af097"} Jul 16 04:47:38.324453 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:38.324021 2581 scope.go:117] "RemoveContainer" containerID="3737631eab03aa87327b380a83d5220343cc7a453c811925d8795d38c1905c51" Jul 16 04:47:38.332451 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:38.332435 2581 scope.go:117] "RemoveContainer" containerID="3737631eab03aa87327b380a83d5220343cc7a453c811925d8795d38c1905c51" Jul 16 04:47:38.332720 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:47:38.332699 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3737631eab03aa87327b380a83d5220343cc7a453c811925d8795d38c1905c51\": container with ID starting with 3737631eab03aa87327b380a83d5220343cc7a453c811925d8795d38c1905c51 not found: ID does not exist" containerID="3737631eab03aa87327b380a83d5220343cc7a453c811925d8795d38c1905c51" Jul 16 04:47:38.332781 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:38.332729 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3737631eab03aa87327b380a83d5220343cc7a453c811925d8795d38c1905c51"} err="failed to get container status \"3737631eab03aa87327b380a83d5220343cc7a453c811925d8795d38c1905c51\": rpc error: code = NotFound desc = could not find container \"3737631eab03aa87327b380a83d5220343cc7a453c811925d8795d38c1905c51\": container with ID starting with 3737631eab03aa87327b380a83d5220343cc7a453c811925d8795d38c1905c51 not found: ID does not exist" Jul 16 04:47:38.347068 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:38.347044 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jkv82"] Jul 16 04:47:38.353293 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:38.353271 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jkv82"] Jul 16 04:47:39.838807 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:39.838775 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b463b567-fa2d-4221-bed0-01e505b97504" path="/var/lib/kubelet/pods/b463b567-fa2d-4221-bed0-01e505b97504/volumes" Jul 16 04:47:43.634899 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:43.634859 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:47:43.734833 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:43.734797 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:47:43.941477 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:43.941387 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-c9c47"] Jul 16 04:47:43.941800 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:43.941783 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b463b567-fa2d-4221-bed0-01e505b97504" containerName="limitador" Jul 16 04:47:43.941895 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:43.941802 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b463b567-fa2d-4221-bed0-01e505b97504" containerName="limitador" Jul 16 04:47:43.941895 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:43.941884 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b463b567-fa2d-4221-bed0-01e505b97504" containerName="limitador" Jul 16 04:47:43.946292 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:43.946271 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c9c47" Jul 16 04:47:43.949214 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:43.949186 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 16 04:47:43.949325 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:43.949241 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-jpsg7\"" Jul 16 04:47:43.949325 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:43.949301 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 16 04:47:43.949438 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:43.949340 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 04:47:43.952154 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:43.952132 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-c9c47"] Jul 16 04:47:43.971113 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:43.971089 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:47:44.104209 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:44.104177 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqtdx\" (UniqueName: \"kubernetes.io/projected/af0cf299-db3c-4a03-b0c4-33e36c883a33-kube-api-access-lqtdx\") pod \"postgres-69fb4594cf-c9c47\" (UID: \"af0cf299-db3c-4a03-b0c4-33e36c883a33\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-c9c47" Jul 16 04:47:44.104367 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:44.104237 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/af0cf299-db3c-4a03-b0c4-33e36c883a33-data\") pod \"postgres-69fb4594cf-c9c47\" (UID: \"af0cf299-db3c-4a03-b0c4-33e36c883a33\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-c9c47" Jul 16 04:47:44.205691 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:44.205612 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/af0cf299-db3c-4a03-b0c4-33e36c883a33-data\") pod \"postgres-69fb4594cf-c9c47\" (UID: \"af0cf299-db3c-4a03-b0c4-33e36c883a33\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-c9c47" Jul 16 04:47:44.205691 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:44.205682 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqtdx\" (UniqueName: \"kubernetes.io/projected/af0cf299-db3c-4a03-b0c4-33e36c883a33-kube-api-access-lqtdx\") pod \"postgres-69fb4594cf-c9c47\" (UID: \"af0cf299-db3c-4a03-b0c4-33e36c883a33\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-c9c47" Jul 16 04:47:44.205981 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:44.205961 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/af0cf299-db3c-4a03-b0c4-33e36c883a33-data\") pod \"postgres-69fb4594cf-c9c47\" (UID: \"af0cf299-db3c-4a03-b0c4-33e36c883a33\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-c9c47" Jul 16 04:47:44.223645 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:44.223617 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqtdx\" (UniqueName: \"kubernetes.io/projected/af0cf299-db3c-4a03-b0c4-33e36c883a33-kube-api-access-lqtdx\") pod \"postgres-69fb4594cf-c9c47\" (UID: \"af0cf299-db3c-4a03-b0c4-33e36c883a33\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-c9c47" Jul 16 04:47:44.257945 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:44.257901 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c9c47" Jul 16 04:47:44.389423 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:44.389331 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-c9c47"] Jul 16 04:47:44.391996 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:47:44.391967 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf0cf299_db3c_4a03_b0c4_33e36c883a33.slice/crio-bf5e3daba9877c5a82275464cfd660d930483b4e298e09dda161409674ebd23e WatchSource:0}: Error finding container bf5e3daba9877c5a82275464cfd660d930483b4e298e09dda161409674ebd23e: Status 404 returned error can't find the container with id bf5e3daba9877c5a82275464cfd660d930483b4e298e09dda161409674ebd23e Jul 16 04:47:45.368558 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:45.368517 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c9c47" event={"ID":"af0cf299-db3c-4a03-b0c4-33e36c883a33","Type":"ContainerStarted","Data":"bf5e3daba9877c5a82275464cfd660d930483b4e298e09dda161409674ebd23e"} Jul 16 04:47:49.393313 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:49.393225 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c9c47" event={"ID":"af0cf299-db3c-4a03-b0c4-33e36c883a33","Type":"ContainerStarted","Data":"6449253508e55872d68c70a365c816b7d045c2154aaf7beb649eea87c8e23499"} Jul 16 04:47:49.393313 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:49.393303 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c9c47" Jul 16 04:47:49.411095 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:49.411047 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c9c47" podStartSLOduration=1.7359141949999999 podStartE2EDuration="6.411032372s" totalImagesPullingTime="4.675118177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:47:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:47:49.4091144 +0000 UTC m=+522.134582919" watchObservedRunningTime="2026-07-16 04:47:49.411032372 +0000 UTC m=+522.136500893" Jul 16 04:47:55.426877 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:55.426848 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c9c47" Jul 16 04:47:55.756836 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:55.756785 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:47:55.856669 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:55.856638 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:47:56.242711 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.242678 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d8f477f69-2d7h8"] Jul 16 04:47:56.247819 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.247801 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8f477f69-2d7h8" Jul 16 04:47:56.253451 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.253424 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d8f477f69-2d7h8"] Jul 16 04:47:56.309869 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.309836 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d78gt\" (UniqueName: \"kubernetes.io/projected/96a2415d-130a-4a2b-809e-0dffc7a1ab34-kube-api-access-d78gt\") pod \"authorino-d8f477f69-2d7h8\" (UID: \"96a2415d-130a-4a2b-809e-0dffc7a1ab34\") " pod="kuadrant-system/authorino-d8f477f69-2d7h8" Jul 16 04:47:56.410580 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.410547 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d78gt\" (UniqueName: \"kubernetes.io/projected/96a2415d-130a-4a2b-809e-0dffc7a1ab34-kube-api-access-d78gt\") pod \"authorino-d8f477f69-2d7h8\" (UID: \"96a2415d-130a-4a2b-809e-0dffc7a1ab34\") " pod="kuadrant-system/authorino-d8f477f69-2d7h8" Jul 16 04:47:56.419100 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.419065 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d78gt\" (UniqueName: \"kubernetes.io/projected/96a2415d-130a-4a2b-809e-0dffc7a1ab34-kube-api-access-d78gt\") pod \"authorino-d8f477f69-2d7h8\" (UID: \"96a2415d-130a-4a2b-809e-0dffc7a1ab34\") " pod="kuadrant-system/authorino-d8f477f69-2d7h8" Jul 16 04:47:56.427028 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.427001 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d8f477f69-2d7h8"] Jul 16 04:47:56.427395 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.427314 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8f477f69-2d7h8" Jul 16 04:47:56.455172 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.455143 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-64c99c69b5-6cf8c"] Jul 16 04:47:56.459492 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.459465 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64c99c69b5-6cf8c" Jul 16 04:47:56.462115 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.462086 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 16 04:47:56.468639 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.468606 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64c99c69b5-6cf8c"] Jul 16 04:47:56.511077 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.510993 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fttsc\" (UniqueName: \"kubernetes.io/projected/225461bb-ee5a-44f5-ae9e-04c612557e9d-kube-api-access-fttsc\") pod \"authorino-64c99c69b5-6cf8c\" (UID: \"225461bb-ee5a-44f5-ae9e-04c612557e9d\") " pod="kuadrant-system/authorino-64c99c69b5-6cf8c" Jul 16 04:47:56.511208 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.511082 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/225461bb-ee5a-44f5-ae9e-04c612557e9d-tls-cert\") pod \"authorino-64c99c69b5-6cf8c\" (UID: \"225461bb-ee5a-44f5-ae9e-04c612557e9d\") " pod="kuadrant-system/authorino-64c99c69b5-6cf8c" Jul 16 04:47:56.564484 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.564443 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d8f477f69-2d7h8"] Jul 16 04:47:56.567374 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:47:56.567342 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod96a2415d_130a_4a2b_809e_0dffc7a1ab34.slice/crio-915b5c416ba256d0f0cf253f388396b1c9dcb4d183c0b815c420e8c379ccd145 WatchSource:0}: Error finding container 915b5c416ba256d0f0cf253f388396b1c9dcb4d183c0b815c420e8c379ccd145: Status 404 returned error can't find the container with id 915b5c416ba256d0f0cf253f388396b1c9dcb4d183c0b815c420e8c379ccd145 Jul 16 04:47:56.611910 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.611869 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/225461bb-ee5a-44f5-ae9e-04c612557e9d-tls-cert\") pod \"authorino-64c99c69b5-6cf8c\" (UID: \"225461bb-ee5a-44f5-ae9e-04c612557e9d\") " pod="kuadrant-system/authorino-64c99c69b5-6cf8c" Jul 16 04:47:56.612104 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.611925 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fttsc\" (UniqueName: \"kubernetes.io/projected/225461bb-ee5a-44f5-ae9e-04c612557e9d-kube-api-access-fttsc\") pod \"authorino-64c99c69b5-6cf8c\" (UID: \"225461bb-ee5a-44f5-ae9e-04c612557e9d\") " pod="kuadrant-system/authorino-64c99c69b5-6cf8c" Jul 16 04:47:56.614291 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.614271 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/225461bb-ee5a-44f5-ae9e-04c612557e9d-tls-cert\") pod \"authorino-64c99c69b5-6cf8c\" (UID: \"225461bb-ee5a-44f5-ae9e-04c612557e9d\") " pod="kuadrant-system/authorino-64c99c69b5-6cf8c" Jul 16 04:47:56.622381 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.622354 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fttsc\" (UniqueName: \"kubernetes.io/projected/225461bb-ee5a-44f5-ae9e-04c612557e9d-kube-api-access-fttsc\") pod \"authorino-64c99c69b5-6cf8c\" (UID: \"225461bb-ee5a-44f5-ae9e-04c612557e9d\") " pod="kuadrant-system/authorino-64c99c69b5-6cf8c" Jul 16 04:47:56.772075 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.771995 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64c99c69b5-6cf8c" Jul 16 04:47:56.901639 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:56.901616 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64c99c69b5-6cf8c"] Jul 16 04:47:56.905157 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:47:56.905134 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod225461bb_ee5a_44f5_ae9e_04c612557e9d.slice/crio-deb6c2e5eff15a1219a3870fb11e64d96c8e16129514c8c9c2c36b3e1397b562 WatchSource:0}: Error finding container deb6c2e5eff15a1219a3870fb11e64d96c8e16129514c8c9c2c36b3e1397b562: Status 404 returned error can't find the container with id deb6c2e5eff15a1219a3870fb11e64d96c8e16129514c8c9c2c36b3e1397b562 Jul 16 04:47:57.434035 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:57.433990 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64c99c69b5-6cf8c" event={"ID":"225461bb-ee5a-44f5-ae9e-04c612557e9d","Type":"ContainerStarted","Data":"38a20803ab736b6375b9621c65fd8625c09d981a931663ba6243bd7d858deadc"} Jul 16 04:47:57.434035 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:57.434037 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64c99c69b5-6cf8c" event={"ID":"225461bb-ee5a-44f5-ae9e-04c612557e9d","Type":"ContainerStarted","Data":"deb6c2e5eff15a1219a3870fb11e64d96c8e16129514c8c9c2c36b3e1397b562"} Jul 16 04:47:57.435761 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:57.435730 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8f477f69-2d7h8" event={"ID":"96a2415d-130a-4a2b-809e-0dffc7a1ab34","Type":"ContainerStarted","Data":"41a0ee807de5a6b577746bdf353a0edefaf86326804f66371953bfb603cbac48"} Jul 16 04:47:57.435876 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:57.435764 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8f477f69-2d7h8" event={"ID":"96a2415d-130a-4a2b-809e-0dffc7a1ab34","Type":"ContainerStarted","Data":"915b5c416ba256d0f0cf253f388396b1c9dcb4d183c0b815c420e8c379ccd145"} Jul 16 04:47:57.435876 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:57.435786 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-d8f477f69-2d7h8" podUID="96a2415d-130a-4a2b-809e-0dffc7a1ab34" containerName="authorino" containerID="cri-o://41a0ee807de5a6b577746bdf353a0edefaf86326804f66371953bfb603cbac48" gracePeriod=30 Jul 16 04:47:57.451408 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:57.451369 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-64c99c69b5-6cf8c" podStartSLOduration=1.097958113 podStartE2EDuration="1.451353896s" totalImagesPullingTime="353.395783ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:47:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:47:57.450031295 +0000 UTC m=+530.175499818" watchObservedRunningTime="2026-07-16 04:47:57.451353896 +0000 UTC m=+530.176822418" Jul 16 04:47:57.465915 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:57.465872 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-d8f477f69-2d7h8" podStartSLOduration=1.139490139 podStartE2EDuration="1.465857206s" totalImagesPullingTime="326.367067ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:47:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:47:57.463562751 +0000 UTC m=+530.189031272" watchObservedRunningTime="2026-07-16 04:47:57.465857206 +0000 UTC m=+530.191325727" Jul 16 04:47:57.481772 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:57.481747 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wwzng"] Jul 16 04:47:57.481944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:57.481923 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-wwzng" podUID="742209b3-9108-40b4-b2cf-fb2b5f30510c" containerName="authorino" containerID="cri-o://f39b38665e79db027b7d663d2ce447c2a6ff32769110d09746add1cdaeeb4f93" gracePeriod=30 Jul 16 04:47:57.726258 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:57.726235 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8f477f69-2d7h8" Jul 16 04:47:57.742641 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:57.742615 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wwzng" Jul 16 04:47:57.821786 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:57.821757 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bl4hs\" (UniqueName: \"kubernetes.io/projected/742209b3-9108-40b4-b2cf-fb2b5f30510c-kube-api-access-bl4hs\") pod \"742209b3-9108-40b4-b2cf-fb2b5f30510c\" (UID: \"742209b3-9108-40b4-b2cf-fb2b5f30510c\") " Jul 16 04:47:57.821786 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:57.821790 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d78gt\" (UniqueName: \"kubernetes.io/projected/96a2415d-130a-4a2b-809e-0dffc7a1ab34-kube-api-access-d78gt\") pod \"96a2415d-130a-4a2b-809e-0dffc7a1ab34\" (UID: \"96a2415d-130a-4a2b-809e-0dffc7a1ab34\") " Jul 16 04:47:57.823893 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:57.823866 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96a2415d-130a-4a2b-809e-0dffc7a1ab34-kube-api-access-d78gt" (OuterVolumeSpecName: "kube-api-access-d78gt") pod "96a2415d-130a-4a2b-809e-0dffc7a1ab34" (UID: "96a2415d-130a-4a2b-809e-0dffc7a1ab34"). InnerVolumeSpecName "kube-api-access-d78gt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:47:57.823996 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:57.823920 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/742209b3-9108-40b4-b2cf-fb2b5f30510c-kube-api-access-bl4hs" (OuterVolumeSpecName: "kube-api-access-bl4hs") pod "742209b3-9108-40b4-b2cf-fb2b5f30510c" (UID: "742209b3-9108-40b4-b2cf-fb2b5f30510c"). InnerVolumeSpecName "kube-api-access-bl4hs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:47:57.923116 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:57.923081 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bl4hs\" (UniqueName: \"kubernetes.io/projected/742209b3-9108-40b4-b2cf-fb2b5f30510c-kube-api-access-bl4hs\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:47:57.923116 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:57.923110 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d78gt\" (UniqueName: \"kubernetes.io/projected/96a2415d-130a-4a2b-809e-0dffc7a1ab34-kube-api-access-d78gt\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:47:58.441336 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.441300 2581 generic.go:356] "Generic (PLEG): container finished" podID="742209b3-9108-40b4-b2cf-fb2b5f30510c" containerID="f39b38665e79db027b7d663d2ce447c2a6ff32769110d09746add1cdaeeb4f93" exitCode=0 Jul 16 04:47:58.441772 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.441339 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wwzng" Jul 16 04:47:58.441772 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.441390 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wwzng" event={"ID":"742209b3-9108-40b4-b2cf-fb2b5f30510c","Type":"ContainerDied","Data":"f39b38665e79db027b7d663d2ce447c2a6ff32769110d09746add1cdaeeb4f93"} Jul 16 04:47:58.441772 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.441440 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wwzng" event={"ID":"742209b3-9108-40b4-b2cf-fb2b5f30510c","Type":"ContainerDied","Data":"a84c8091d11648c16509fffbddbefd20c7ae22695908a677cceaec53e3d52e72"} Jul 16 04:47:58.441772 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.441466 2581 scope.go:117] "RemoveContainer" containerID="f39b38665e79db027b7d663d2ce447c2a6ff32769110d09746add1cdaeeb4f93" Jul 16 04:47:58.442829 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.442805 2581 generic.go:356] "Generic (PLEG): container finished" podID="96a2415d-130a-4a2b-809e-0dffc7a1ab34" containerID="41a0ee807de5a6b577746bdf353a0edefaf86326804f66371953bfb603cbac48" exitCode=0 Jul 16 04:47:58.442928 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.442845 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8f477f69-2d7h8" Jul 16 04:47:58.442928 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.442892 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8f477f69-2d7h8" event={"ID":"96a2415d-130a-4a2b-809e-0dffc7a1ab34","Type":"ContainerDied","Data":"41a0ee807de5a6b577746bdf353a0edefaf86326804f66371953bfb603cbac48"} Jul 16 04:47:58.443037 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.442929 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8f477f69-2d7h8" event={"ID":"96a2415d-130a-4a2b-809e-0dffc7a1ab34","Type":"ContainerDied","Data":"915b5c416ba256d0f0cf253f388396b1c9dcb4d183c0b815c420e8c379ccd145"} Jul 16 04:47:58.450360 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.450343 2581 scope.go:117] "RemoveContainer" containerID="f39b38665e79db027b7d663d2ce447c2a6ff32769110d09746add1cdaeeb4f93" Jul 16 04:47:58.450632 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:47:58.450613 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f39b38665e79db027b7d663d2ce447c2a6ff32769110d09746add1cdaeeb4f93\": container with ID starting with f39b38665e79db027b7d663d2ce447c2a6ff32769110d09746add1cdaeeb4f93 not found: ID does not exist" containerID="f39b38665e79db027b7d663d2ce447c2a6ff32769110d09746add1cdaeeb4f93" Jul 16 04:47:58.450712 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.450643 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f39b38665e79db027b7d663d2ce447c2a6ff32769110d09746add1cdaeeb4f93"} err="failed to get container status \"f39b38665e79db027b7d663d2ce447c2a6ff32769110d09746add1cdaeeb4f93\": rpc error: code = NotFound desc = could not find container \"f39b38665e79db027b7d663d2ce447c2a6ff32769110d09746add1cdaeeb4f93\": container with ID starting with f39b38665e79db027b7d663d2ce447c2a6ff32769110d09746add1cdaeeb4f93 not found: ID does not exist" Jul 16 04:47:58.450712 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.450664 2581 scope.go:117] "RemoveContainer" containerID="41a0ee807de5a6b577746bdf353a0edefaf86326804f66371953bfb603cbac48" Jul 16 04:47:58.458495 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.458479 2581 scope.go:117] "RemoveContainer" containerID="41a0ee807de5a6b577746bdf353a0edefaf86326804f66371953bfb603cbac48" Jul 16 04:47:58.458776 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:47:58.458759 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41a0ee807de5a6b577746bdf353a0edefaf86326804f66371953bfb603cbac48\": container with ID starting with 41a0ee807de5a6b577746bdf353a0edefaf86326804f66371953bfb603cbac48 not found: ID does not exist" containerID="41a0ee807de5a6b577746bdf353a0edefaf86326804f66371953bfb603cbac48" Jul 16 04:47:58.458831 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.458782 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41a0ee807de5a6b577746bdf353a0edefaf86326804f66371953bfb603cbac48"} err="failed to get container status \"41a0ee807de5a6b577746bdf353a0edefaf86326804f66371953bfb603cbac48\": rpc error: code = NotFound desc = could not find container \"41a0ee807de5a6b577746bdf353a0edefaf86326804f66371953bfb603cbac48\": container with ID starting with 41a0ee807de5a6b577746bdf353a0edefaf86326804f66371953bfb603cbac48 not found: ID does not exist" Jul 16 04:47:58.459967 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.459949 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wwzng"] Jul 16 04:47:58.463557 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.463535 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wwzng"] Jul 16 04:47:58.473108 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.473087 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d8f477f69-2d7h8"] Jul 16 04:47:58.478003 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.477983 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-d8f477f69-2d7h8"] Jul 16 04:47:58.896289 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:58.896254 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:47:59.838351 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:59.838317 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="742209b3-9108-40b4-b2cf-fb2b5f30510c" path="/var/lib/kubelet/pods/742209b3-9108-40b4-b2cf-fb2b5f30510c/volumes" Jul 16 04:47:59.838776 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:47:59.838759 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96a2415d-130a-4a2b-809e-0dffc7a1ab34" path="/var/lib/kubelet/pods/96a2415d-130a-4a2b-809e-0dffc7a1ab34/volumes" Jul 16 04:48:02.185016 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.184972 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:48:02.194283 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.194259 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d9cb48bd8-x7mbc"] Jul 16 04:48:02.194631 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.194620 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="742209b3-9108-40b4-b2cf-fb2b5f30510c" containerName="authorino" Jul 16 04:48:02.194671 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.194633 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="742209b3-9108-40b4-b2cf-fb2b5f30510c" containerName="authorino" Jul 16 04:48:02.194671 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.194650 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96a2415d-130a-4a2b-809e-0dffc7a1ab34" containerName="authorino" Jul 16 04:48:02.194671 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.194656 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="96a2415d-130a-4a2b-809e-0dffc7a1ab34" containerName="authorino" Jul 16 04:48:02.194763 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.194708 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="96a2415d-130a-4a2b-809e-0dffc7a1ab34" containerName="authorino" Jul 16 04:48:02.194763 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.194719 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="742209b3-9108-40b4-b2cf-fb2b5f30510c" containerName="authorino" Jul 16 04:48:02.236311 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.236273 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d9cb48bd8-x7mbc"] Jul 16 04:48:02.236461 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.236378 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d9cb48bd8-x7mbc" Jul 16 04:48:02.240386 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.240367 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 16 04:48:02.240583 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.240367 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 16 04:48:02.240583 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.240367 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6ds6k\"" Jul 16 04:48:02.260532 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.260492 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3d32d01b-d0e9-4f15-929f-a01579e79350-webhook-cert\") pod \"maas-controller-7d9cb48bd8-x7mbc\" (UID: \"3d32d01b-d0e9-4f15-929f-a01579e79350\") " pod="opendatahub/maas-controller-7d9cb48bd8-x7mbc" Jul 16 04:48:02.260622 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.260571 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzntq\" (UniqueName: \"kubernetes.io/projected/3d32d01b-d0e9-4f15-929f-a01579e79350-kube-api-access-dzntq\") pod \"maas-controller-7d9cb48bd8-x7mbc\" (UID: \"3d32d01b-d0e9-4f15-929f-a01579e79350\") " pod="opendatahub/maas-controller-7d9cb48bd8-x7mbc" Jul 16 04:48:02.262229 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.262208 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:48:02.341555 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.341498 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:48:02.361690 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.361664 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3d32d01b-d0e9-4f15-929f-a01579e79350-webhook-cert\") pod \"maas-controller-7d9cb48bd8-x7mbc\" (UID: \"3d32d01b-d0e9-4f15-929f-a01579e79350\") " pod="opendatahub/maas-controller-7d9cb48bd8-x7mbc" Jul 16 04:48:02.361821 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.361711 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzntq\" (UniqueName: \"kubernetes.io/projected/3d32d01b-d0e9-4f15-929f-a01579e79350-kube-api-access-dzntq\") pod \"maas-controller-7d9cb48bd8-x7mbc\" (UID: \"3d32d01b-d0e9-4f15-929f-a01579e79350\") " pod="opendatahub/maas-controller-7d9cb48bd8-x7mbc" Jul 16 04:48:02.364163 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.364142 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3d32d01b-d0e9-4f15-929f-a01579e79350-webhook-cert\") pod \"maas-controller-7d9cb48bd8-x7mbc\" (UID: \"3d32d01b-d0e9-4f15-929f-a01579e79350\") " pod="opendatahub/maas-controller-7d9cb48bd8-x7mbc" Jul 16 04:48:02.373824 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.373802 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzntq\" (UniqueName: \"kubernetes.io/projected/3d32d01b-d0e9-4f15-929f-a01579e79350-kube-api-access-dzntq\") pod \"maas-controller-7d9cb48bd8-x7mbc\" (UID: \"3d32d01b-d0e9-4f15-929f-a01579e79350\") " pod="opendatahub/maas-controller-7d9cb48bd8-x7mbc" Jul 16 04:48:02.546466 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.546429 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d9cb48bd8-x7mbc" Jul 16 04:48:02.710254 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:02.710227 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d9cb48bd8-x7mbc"] Jul 16 04:48:02.712175 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:48:02.712145 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d32d01b_d0e9_4f15_929f_a01579e79350.slice/crio-9fc82acf4b8a80cf135a979ee84b205c3db5de922b463ceb74b6bd84cbe23e3a WatchSource:0}: Error finding container 9fc82acf4b8a80cf135a979ee84b205c3db5de922b463ceb74b6bd84cbe23e3a: Status 404 returned error can't find the container with id 9fc82acf4b8a80cf135a979ee84b205c3db5de922b463ceb74b6bd84cbe23e3a Jul 16 04:48:03.471828 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:03.471793 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d9cb48bd8-x7mbc" event={"ID":"3d32d01b-d0e9-4f15-929f-a01579e79350","Type":"ContainerStarted","Data":"9fc82acf4b8a80cf135a979ee84b205c3db5de922b463ceb74b6bd84cbe23e3a"} Jul 16 04:48:05.454458 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:05.454427 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:48:05.484193 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:05.484159 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d9cb48bd8-x7mbc" event={"ID":"3d32d01b-d0e9-4f15-929f-a01579e79350","Type":"ContainerStarted","Data":"97cf5848278de992b32a25b6face5526d0a5ec40efffb28161053e8e9991a546"} Jul 16 04:48:05.484335 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:05.484268 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d9cb48bd8-x7mbc" Jul 16 04:48:05.525911 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:05.525818 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d9cb48bd8-x7mbc" podStartSLOduration=0.988797003 podStartE2EDuration="3.525803907s" totalImagesPullingTime="2.537006904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:48:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:48:05.521484088 +0000 UTC m=+538.246952610" watchObservedRunningTime="2026-07-16 04:48:05.525803907 +0000 UTC m=+538.251272428" Jul 16 04:48:05.590287 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:05.590253 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:48:16.491263 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:16.491232 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d9cb48bd8-x7mbc" Jul 16 04:48:17.164830 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.164798 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d9cb48bd8-x7mbc"] Jul 16 04:48:17.164992 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.164969 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7d9cb48bd8-x7mbc" podUID="3d32d01b-d0e9-4f15-929f-a01579e79350" containerName="manager" containerID="cri-o://97cf5848278de992b32a25b6face5526d0a5ec40efffb28161053e8e9991a546" gracePeriod=30 Jul 16 04:48:17.424299 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.424248 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d9cb48bd8-x7mbc" Jul 16 04:48:17.487019 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.486983 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzntq\" (UniqueName: \"kubernetes.io/projected/3d32d01b-d0e9-4f15-929f-a01579e79350-kube-api-access-dzntq\") pod \"3d32d01b-d0e9-4f15-929f-a01579e79350\" (UID: \"3d32d01b-d0e9-4f15-929f-a01579e79350\") " Jul 16 04:48:17.487152 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.487104 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3d32d01b-d0e9-4f15-929f-a01579e79350-webhook-cert\") pod \"3d32d01b-d0e9-4f15-929f-a01579e79350\" (UID: \"3d32d01b-d0e9-4f15-929f-a01579e79350\") " Jul 16 04:48:17.489209 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.489178 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d32d01b-d0e9-4f15-929f-a01579e79350-kube-api-access-dzntq" (OuterVolumeSpecName: "kube-api-access-dzntq") pod "3d32d01b-d0e9-4f15-929f-a01579e79350" (UID: "3d32d01b-d0e9-4f15-929f-a01579e79350"). InnerVolumeSpecName "kube-api-access-dzntq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:48:17.489209 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.489193 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d32d01b-d0e9-4f15-929f-a01579e79350-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3d32d01b-d0e9-4f15-929f-a01579e79350" (UID: "3d32d01b-d0e9-4f15-929f-a01579e79350"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:48:17.541218 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.541194 2581 generic.go:356] "Generic (PLEG): container finished" podID="3d32d01b-d0e9-4f15-929f-a01579e79350" containerID="97cf5848278de992b32a25b6face5526d0a5ec40efffb28161053e8e9991a546" exitCode=0 Jul 16 04:48:17.541536 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.541238 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d9cb48bd8-x7mbc" Jul 16 04:48:17.541536 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.541287 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d9cb48bd8-x7mbc" event={"ID":"3d32d01b-d0e9-4f15-929f-a01579e79350","Type":"ContainerDied","Data":"97cf5848278de992b32a25b6face5526d0a5ec40efffb28161053e8e9991a546"} Jul 16 04:48:17.541536 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.541341 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d9cb48bd8-x7mbc" event={"ID":"3d32d01b-d0e9-4f15-929f-a01579e79350","Type":"ContainerDied","Data":"9fc82acf4b8a80cf135a979ee84b205c3db5de922b463ceb74b6bd84cbe23e3a"} Jul 16 04:48:17.541536 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.541369 2581 scope.go:117] "RemoveContainer" containerID="97cf5848278de992b32a25b6face5526d0a5ec40efffb28161053e8e9991a546" Jul 16 04:48:17.550137 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.550121 2581 scope.go:117] "RemoveContainer" containerID="97cf5848278de992b32a25b6face5526d0a5ec40efffb28161053e8e9991a546" Jul 16 04:48:17.550385 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:48:17.550366 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97cf5848278de992b32a25b6face5526d0a5ec40efffb28161053e8e9991a546\": container with ID starting with 97cf5848278de992b32a25b6face5526d0a5ec40efffb28161053e8e9991a546 not found: ID does not exist" containerID="97cf5848278de992b32a25b6face5526d0a5ec40efffb28161053e8e9991a546" Jul 16 04:48:17.550442 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.550394 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97cf5848278de992b32a25b6face5526d0a5ec40efffb28161053e8e9991a546"} err="failed to get container status \"97cf5848278de992b32a25b6face5526d0a5ec40efffb28161053e8e9991a546\": rpc error: code = NotFound desc = could not find container \"97cf5848278de992b32a25b6face5526d0a5ec40efffb28161053e8e9991a546\": container with ID starting with 97cf5848278de992b32a25b6face5526d0a5ec40efffb28161053e8e9991a546 not found: ID does not exist" Jul 16 04:48:17.562741 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.562718 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d9cb48bd8-x7mbc"] Jul 16 04:48:17.566629 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.566610 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7d9cb48bd8-x7mbc"] Jul 16 04:48:17.588212 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.588193 2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3d32d01b-d0e9-4f15-929f-a01579e79350-webhook-cert\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:48:17.588294 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.588216 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzntq\" (UniqueName: \"kubernetes.io/projected/3d32d01b-d0e9-4f15-929f-a01579e79350-kube-api-access-dzntq\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:48:17.839150 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:17.839118 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d32d01b-d0e9-4f15-929f-a01579e79350" path="/var/lib/kubelet/pods/3d32d01b-d0e9-4f15-929f-a01579e79350/volumes" Jul 16 04:48:37.447325 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:37.447281 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:48:37.543608 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:37.543574 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:48:38.006209 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.006163 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:48:38.085553 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.085518 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm"] Jul 16 04:48:38.086056 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.086042 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d32d01b-d0e9-4f15-929f-a01579e79350" containerName="manager" Jul 16 04:48:38.086100 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.086059 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d32d01b-d0e9-4f15-929f-a01579e79350" containerName="manager" Jul 16 04:48:38.086168 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.086157 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d32d01b-d0e9-4f15-929f-a01579e79350" containerName="manager" Jul 16 04:48:38.091070 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.091049 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm" Jul 16 04:48:38.094040 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.094023 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 16 04:48:38.108091 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.108067 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm"] Jul 16 04:48:38.159768 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.159739 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89cea744-972e-4854-bfcc-19575fb43e30-maas-api-tls\") pod \"maas-api-76569bc6fb-p9sjm\" (UID: \"89cea744-972e-4854-bfcc-19575fb43e30\") " pod="odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm" Jul 16 04:48:38.159768 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.159770 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89cea744-972e-4854-bfcc-19575fb43e30-openshift-service-ca\") pod \"maas-api-76569bc6fb-p9sjm\" (UID: \"89cea744-972e-4854-bfcc-19575fb43e30\") " pod="odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm" Jul 16 04:48:38.160190 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.159795 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsmgs\" (UniqueName: \"kubernetes.io/projected/89cea744-972e-4854-bfcc-19575fb43e30-kube-api-access-qsmgs\") pod \"maas-api-76569bc6fb-p9sjm\" (UID: \"89cea744-972e-4854-bfcc-19575fb43e30\") " pod="odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm" Jul 16 04:48:38.245949 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.245912 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:48:38.260756 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.260691 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89cea744-972e-4854-bfcc-19575fb43e30-maas-api-tls\") pod \"maas-api-76569bc6fb-p9sjm\" (UID: \"89cea744-972e-4854-bfcc-19575fb43e30\") " pod="odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm" Jul 16 04:48:38.260756 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.260719 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89cea744-972e-4854-bfcc-19575fb43e30-openshift-service-ca\") pod \"maas-api-76569bc6fb-p9sjm\" (UID: \"89cea744-972e-4854-bfcc-19575fb43e30\") " pod="odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm" Jul 16 04:48:38.260756 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.260743 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsmgs\" (UniqueName: \"kubernetes.io/projected/89cea744-972e-4854-bfcc-19575fb43e30-kube-api-access-qsmgs\") pod \"maas-api-76569bc6fb-p9sjm\" (UID: \"89cea744-972e-4854-bfcc-19575fb43e30\") " pod="odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm" Jul 16 04:48:38.261377 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.261356 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89cea744-972e-4854-bfcc-19575fb43e30-openshift-service-ca\") pod \"maas-api-76569bc6fb-p9sjm\" (UID: \"89cea744-972e-4854-bfcc-19575fb43e30\") " pod="odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm" Jul 16 04:48:38.263131 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.263113 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89cea744-972e-4854-bfcc-19575fb43e30-maas-api-tls\") pod \"maas-api-76569bc6fb-p9sjm\" (UID: \"89cea744-972e-4854-bfcc-19575fb43e30\") " pod="odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm" Jul 16 04:48:38.270378 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.270354 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsmgs\" (UniqueName: \"kubernetes.io/projected/89cea744-972e-4854-bfcc-19575fb43e30-kube-api-access-qsmgs\") pod \"maas-api-76569bc6fb-p9sjm\" (UID: \"89cea744-972e-4854-bfcc-19575fb43e30\") " pod="odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm" Jul 16 04:48:38.401571 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.401538 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm" Jul 16 04:48:38.539298 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.539271 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm"] Jul 16 04:48:38.542275 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:48:38.542247 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89cea744_972e_4854_bfcc_19575fb43e30.slice/crio-b0dad811c61075ae679fb0f4306d306eea5d3cece3b24a3d2c285633ab425b48 WatchSource:0}: Error finding container b0dad811c61075ae679fb0f4306d306eea5d3cece3b24a3d2c285633ab425b48: Status 404 returned error can't find the container with id b0dad811c61075ae679fb0f4306d306eea5d3cece3b24a3d2c285633ab425b48 Jul 16 04:48:38.580576 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.580549 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:48:38.639474 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.639427 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm" event={"ID":"89cea744-972e-4854-bfcc-19575fb43e30","Type":"ContainerStarted","Data":"b0dad811c61075ae679fb0f4306d306eea5d3cece3b24a3d2c285633ab425b48"} Jul 16 04:48:38.682030 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.682000 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:48:38.726404 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.726379 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-6cqdj"] Jul 16 04:48:38.729452 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.729436 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-6cqdj" Jul 16 04:48:38.732106 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.732085 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 16 04:48:38.732336 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.732319 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-dt4lc\"" Jul 16 04:48:38.741272 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.741248 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-6cqdj"] Jul 16 04:48:38.865973 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.865895 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8bv8\" (UniqueName: \"kubernetes.io/projected/bbf6a1d3-485d-4840-9920-33d23f331880-kube-api-access-t8bv8\") pod \"payload-processing-f9b8d9b9b-6cqdj\" (UID: \"bbf6a1d3-485d-4840-9920-33d23f331880\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-6cqdj" Jul 16 04:48:38.865973 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.865932 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbf6a1d3-485d-4840-9920-33d23f331880-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-6cqdj\" (UID: \"bbf6a1d3-485d-4840-9920-33d23f331880\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-6cqdj" Jul 16 04:48:38.967056 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.967019 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8bv8\" (UniqueName: \"kubernetes.io/projected/bbf6a1d3-485d-4840-9920-33d23f331880-kube-api-access-t8bv8\") pod \"payload-processing-f9b8d9b9b-6cqdj\" (UID: \"bbf6a1d3-485d-4840-9920-33d23f331880\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-6cqdj" Jul 16 04:48:38.967231 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.967068 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbf6a1d3-485d-4840-9920-33d23f331880-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-6cqdj\" (UID: \"bbf6a1d3-485d-4840-9920-33d23f331880\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-6cqdj" Jul 16 04:48:38.967762 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.967742 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbf6a1d3-485d-4840-9920-33d23f331880-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-6cqdj\" (UID: \"bbf6a1d3-485d-4840-9920-33d23f331880\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-6cqdj" Jul 16 04:48:38.975965 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:38.975942 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8bv8\" (UniqueName: \"kubernetes.io/projected/bbf6a1d3-485d-4840-9920-33d23f331880-kube-api-access-t8bv8\") pod \"payload-processing-f9b8d9b9b-6cqdj\" (UID: \"bbf6a1d3-485d-4840-9920-33d23f331880\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-6cqdj" Jul 16 04:48:39.040734 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:39.040710 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-6cqdj" Jul 16 04:48:39.383811 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:39.380656 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-6cqdj"] Jul 16 04:48:39.384813 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:48:39.384766 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbf6a1d3_485d_4840_9920_33d23f331880.slice/crio-c74e6164b47294fb7676ce571298f0f2399952d69000a561206463f09a31463d WatchSource:0}: Error finding container c74e6164b47294fb7676ce571298f0f2399952d69000a561206463f09a31463d: Status 404 returned error can't find the container with id c74e6164b47294fb7676ce571298f0f2399952d69000a561206463f09a31463d Jul 16 04:48:39.648925 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:39.648813 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-6cqdj" event={"ID":"bbf6a1d3-485d-4840-9920-33d23f331880","Type":"ContainerStarted","Data":"c74e6164b47294fb7676ce571298f0f2399952d69000a561206463f09a31463d"} Jul 16 04:48:41.672153 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:41.672119 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm" event={"ID":"89cea744-972e-4854-bfcc-19575fb43e30","Type":"ContainerStarted","Data":"e4299136894fcf718d8d0745945f887e86bee62fba001283699391a280ff3949"} Jul 16 04:48:41.672632 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:41.672208 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm" Jul 16 04:48:41.673672 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:41.673647 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-6cqdj" event={"ID":"bbf6a1d3-485d-4840-9920-33d23f331880","Type":"ContainerStarted","Data":"4d73127dc653cde0be795d2a6b103cf010a62ab051ca5008311c919b5f9c1686"} Jul 16 04:48:41.673801 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:41.673719 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-6cqdj" Jul 16 04:48:41.693089 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:41.693036 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm" podStartSLOduration=1.147218648 podStartE2EDuration="3.693022092s" totalImagesPullingTime="2.545803444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:48:41.690540839 +0000 UTC m=+574.416009358" watchObservedRunningTime="2026-07-16 04:48:41.693022092 +0000 UTC m=+574.418490614" Jul 16 04:48:41.707682 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:41.707640 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-6cqdj" podStartSLOduration=2.003827187 podStartE2EDuration="3.707625547s" totalImagesPullingTime="1.70379836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:48:41.70666977 +0000 UTC m=+574.432138291" watchObservedRunningTime="2026-07-16 04:48:41.707625547 +0000 UTC m=+574.433094068" Jul 16 04:48:46.703896 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:46.703806 2581 generic.go:356] "Generic (PLEG): container finished" podID="9c241788-0107-4d03-a926-7a4bf3c806db" containerID="504f085deb0f6236c46784519f30efcc09168aa051b98cec5919e688baadff8f" exitCode=0 Jul 16 04:48:46.703896 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:46.703857 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" event={"ID":"9c241788-0107-4d03-a926-7a4bf3c806db","Type":"ContainerDied","Data":"504f085deb0f6236c46784519f30efcc09168aa051b98cec5919e688baadff8f"} Jul 16 04:48:46.704315 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:46.704196 2581 scope.go:117] "RemoveContainer" containerID="504f085deb0f6236c46784519f30efcc09168aa051b98cec5919e688baadff8f" Jul 16 04:48:46.705245 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:46.705216 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 04:48:46.705313 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:46.705285 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 04:48:46.705350 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:46.705317 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 04:48:47.687837 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:47.687805 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-76569bc6fb-p9sjm" Jul 16 04:48:47.711140 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:47.711102 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" event={"ID":"9c241788-0107-4d03-a926-7a4bf3c806db","Type":"ContainerStarted","Data":"53747ac0c1b736785def34499525dfd0320aa373e4d29d7436cb8d664054b3a1"} Jul 16 04:48:48.595486 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:48.595449 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:48:48.596655 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:48.596632 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-llv9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 16 04:48:48.596727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:48.596679 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" podUID="9c241788-0107-4d03-a926-7a4bf3c806db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 16 04:48:48.931944 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:48.931869 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:48:49.042111 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:49.042076 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:48:49.595381 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:49.595347 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-llv9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 16 04:48:49.595602 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:49.595414 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" podUID="9c241788-0107-4d03-a926-7a4bf3c806db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 16 04:48:50.598319 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:50.598272 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-llv9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 16 04:48:50.598713 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:50.598456 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" podUID="9c241788-0107-4d03-a926-7a4bf3c806db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 16 04:48:51.602040 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:51.602009 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:48:51.602476 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:51.602287 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:48:51.603022 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:51.603008 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llv9g" Jul 16 04:48:52.684703 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:52.684672 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-6cqdj" Jul 16 04:48:58.114233 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:48:58.114194 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:49:07.738937 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:07.738908 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrbbv_95a3a732-50cf-47f5-9629-0b3d241d017a/ovn-acl-logging/0.log" Jul 16 04:49:07.740816 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:07.740795 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrbbv_95a3a732-50cf-47f5-9629-0b3d241d017a/ovn-acl-logging/0.log" Jul 16 04:49:13.897441 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:13.897412 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl"] Jul 16 04:49:13.903470 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:13.903453 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:13.906103 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:13.906082 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 16 04:49:13.906471 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:13.906449 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 16 04:49:13.906572 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:13.906482 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 16 04:49:13.906935 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:13.906923 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-q2d59\"" Jul 16 04:49:13.911115 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:13.911094 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl"] Jul 16 04:49:13.926449 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:13.926422 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:49:13.990985 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:13.990958 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7979c811-3f46-4b9b-8367-725521e261dc-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bwrcl\" (UID: \"7979c811-3f46-4b9b-8367-725521e261dc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:13.991121 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:13.991006 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7979c811-3f46-4b9b-8367-725521e261dc-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bwrcl\" (UID: \"7979c811-3f46-4b9b-8367-725521e261dc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:13.991121 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:13.991027 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2spq\" (UniqueName: \"kubernetes.io/projected/7979c811-3f46-4b9b-8367-725521e261dc-kube-api-access-p2spq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bwrcl\" (UID: \"7979c811-3f46-4b9b-8367-725521e261dc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:13.991194 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:13.991122 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7979c811-3f46-4b9b-8367-725521e261dc-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bwrcl\" (UID: \"7979c811-3f46-4b9b-8367-725521e261dc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:13.991275 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:13.991194 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7979c811-3f46-4b9b-8367-725521e261dc-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bwrcl\" (UID: \"7979c811-3f46-4b9b-8367-725521e261dc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:13.991275 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:13.991219 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7979c811-3f46-4b9b-8367-725521e261dc-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bwrcl\" (UID: \"7979c811-3f46-4b9b-8367-725521e261dc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:14.091925 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:14.091889 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7979c811-3f46-4b9b-8367-725521e261dc-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bwrcl\" (UID: \"7979c811-3f46-4b9b-8367-725521e261dc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:14.092120 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:14.091948 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7979c811-3f46-4b9b-8367-725521e261dc-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bwrcl\" (UID: \"7979c811-3f46-4b9b-8367-725521e261dc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:14.092120 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:14.092065 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2spq\" (UniqueName: \"kubernetes.io/projected/7979c811-3f46-4b9b-8367-725521e261dc-kube-api-access-p2spq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bwrcl\" (UID: \"7979c811-3f46-4b9b-8367-725521e261dc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:14.092252 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:14.092132 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7979c811-3f46-4b9b-8367-725521e261dc-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bwrcl\" (UID: \"7979c811-3f46-4b9b-8367-725521e261dc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:14.092252 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:14.092198 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7979c811-3f46-4b9b-8367-725521e261dc-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bwrcl\" (UID: \"7979c811-3f46-4b9b-8367-725521e261dc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:14.092252 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:14.092224 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7979c811-3f46-4b9b-8367-725521e261dc-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bwrcl\" (UID: \"7979c811-3f46-4b9b-8367-725521e261dc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:14.092399 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:14.092292 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7979c811-3f46-4b9b-8367-725521e261dc-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bwrcl\" (UID: \"7979c811-3f46-4b9b-8367-725521e261dc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:14.092453 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:14.092438 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7979c811-3f46-4b9b-8367-725521e261dc-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bwrcl\" (UID: \"7979c811-3f46-4b9b-8367-725521e261dc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:14.092678 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:14.092658 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7979c811-3f46-4b9b-8367-725521e261dc-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bwrcl\" (UID: \"7979c811-3f46-4b9b-8367-725521e261dc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:14.094296 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:14.094270 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7979c811-3f46-4b9b-8367-725521e261dc-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bwrcl\" (UID: \"7979c811-3f46-4b9b-8367-725521e261dc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:14.094622 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:14.094607 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7979c811-3f46-4b9b-8367-725521e261dc-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bwrcl\" (UID: \"7979c811-3f46-4b9b-8367-725521e261dc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:14.100541 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:14.100519 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2spq\" (UniqueName: \"kubernetes.io/projected/7979c811-3f46-4b9b-8367-725521e261dc-kube-api-access-p2spq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bwrcl\" (UID: \"7979c811-3f46-4b9b-8367-725521e261dc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:14.214953 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:14.214923 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:14.351925 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:14.351897 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl"] Jul 16 04:49:14.353396 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:49:14.353367 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7979c811_3f46_4b9b_8367_725521e261dc.slice/crio-7318b8d55406e10945876d544da85700f8fe095411541ba324e3b585902e7e2e WatchSource:0}: Error finding container 7318b8d55406e10945876d544da85700f8fe095411541ba324e3b585902e7e2e: Status 404 returned error can't find the container with id 7318b8d55406e10945876d544da85700f8fe095411541ba324e3b585902e7e2e Jul 16 04:49:14.354760 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:14.354740 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 04:49:14.848370 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:14.848325 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" event={"ID":"7979c811-3f46-4b9b-8367-725521e261dc","Type":"ContainerStarted","Data":"7318b8d55406e10945876d544da85700f8fe095411541ba324e3b585902e7e2e"} Jul 16 04:49:15.854589 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:15.854484 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" event={"ID":"7979c811-3f46-4b9b-8367-725521e261dc","Type":"ContainerStarted","Data":"1355bf3a4025b87288fcf81d11538797683239be58e9a3421f35d14766bf34e6"} Jul 16 04:49:15.875344 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:15.875300 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" podStartSLOduration=1.768559054 podStartE2EDuration="2.875285377s" totalImagesPullingTime="1.106726323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:49:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:49:15.873441012 +0000 UTC m=+608.598909535" watchObservedRunningTime="2026-07-16 04:49:15.875285377 +0000 UTC m=+608.600753899" Jul 16 04:49:19.181293 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:19.181259 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:49:24.215732 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:24.215697 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:24.216130 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:24.215761 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:24.228183 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:24.228159 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:24.907190 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:24.907163 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bwrcl" Jul 16 04:49:25.032330 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:25.032291 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:49:29.030936 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:29.030894 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:49:35.122940 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:35.122847 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:49:43.786559 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.786525 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk"] Jul 16 04:49:43.790377 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.790355 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:43.793014 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.792995 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 16 04:49:43.802857 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.802827 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk"] Jul 16 04:49:43.826818 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.826789 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:49:43.858629 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.858598 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe086561-4ba6-410b-a3af-2d0dd032f572-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gqnlk\" (UID: \"fe086561-4ba6-410b-a3af-2d0dd032f572\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:43.858749 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.858671 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww2zf\" (UniqueName: \"kubernetes.io/projected/fe086561-4ba6-410b-a3af-2d0dd032f572-kube-api-access-ww2zf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gqnlk\" (UID: \"fe086561-4ba6-410b-a3af-2d0dd032f572\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:43.858749 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.858714 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe086561-4ba6-410b-a3af-2d0dd032f572-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gqnlk\" (UID: \"fe086561-4ba6-410b-a3af-2d0dd032f572\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:43.858832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.858772 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe086561-4ba6-410b-a3af-2d0dd032f572-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gqnlk\" (UID: \"fe086561-4ba6-410b-a3af-2d0dd032f572\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:43.858832 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.858797 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe086561-4ba6-410b-a3af-2d0dd032f572-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gqnlk\" (UID: \"fe086561-4ba6-410b-a3af-2d0dd032f572\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:43.858913 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.858833 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe086561-4ba6-410b-a3af-2d0dd032f572-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gqnlk\" (UID: \"fe086561-4ba6-410b-a3af-2d0dd032f572\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:43.959403 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.959366 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww2zf\" (UniqueName: \"kubernetes.io/projected/fe086561-4ba6-410b-a3af-2d0dd032f572-kube-api-access-ww2zf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gqnlk\" (UID: \"fe086561-4ba6-410b-a3af-2d0dd032f572\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:43.959584 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.959422 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe086561-4ba6-410b-a3af-2d0dd032f572-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gqnlk\" (UID: \"fe086561-4ba6-410b-a3af-2d0dd032f572\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:43.959584 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.959448 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe086561-4ba6-410b-a3af-2d0dd032f572-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gqnlk\" (UID: \"fe086561-4ba6-410b-a3af-2d0dd032f572\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:43.959584 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.959463 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe086561-4ba6-410b-a3af-2d0dd032f572-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gqnlk\" (UID: \"fe086561-4ba6-410b-a3af-2d0dd032f572\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:43.959584 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.959478 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe086561-4ba6-410b-a3af-2d0dd032f572-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gqnlk\" (UID: \"fe086561-4ba6-410b-a3af-2d0dd032f572\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:43.959584 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.959541 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe086561-4ba6-410b-a3af-2d0dd032f572-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gqnlk\" (UID: \"fe086561-4ba6-410b-a3af-2d0dd032f572\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:43.959932 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.959897 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe086561-4ba6-410b-a3af-2d0dd032f572-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gqnlk\" (UID: \"fe086561-4ba6-410b-a3af-2d0dd032f572\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:43.959932 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.959921 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe086561-4ba6-410b-a3af-2d0dd032f572-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gqnlk\" (UID: \"fe086561-4ba6-410b-a3af-2d0dd032f572\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:43.960088 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.959938 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe086561-4ba6-410b-a3af-2d0dd032f572-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gqnlk\" (UID: \"fe086561-4ba6-410b-a3af-2d0dd032f572\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:43.961844 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.961818 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe086561-4ba6-410b-a3af-2d0dd032f572-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gqnlk\" (UID: \"fe086561-4ba6-410b-a3af-2d0dd032f572\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:43.962022 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.962005 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe086561-4ba6-410b-a3af-2d0dd032f572-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gqnlk\" (UID: \"fe086561-4ba6-410b-a3af-2d0dd032f572\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:43.966975 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:43.966950 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww2zf\" (UniqueName: \"kubernetes.io/projected/fe086561-4ba6-410b-a3af-2d0dd032f572-kube-api-access-ww2zf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gqnlk\" (UID: \"fe086561-4ba6-410b-a3af-2d0dd032f572\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:44.106166 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:44.106073 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:44.236945 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:44.236915 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk"] Jul 16 04:49:44.238877 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:49:44.238847 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe086561_4ba6_410b_a3af_2d0dd032f572.slice/crio-7f0b602b2e26453dce9d4b6b0a8af2dae352cf56585f205b3b0f9c6c93b15b2b WatchSource:0}: Error finding container 7f0b602b2e26453dce9d4b6b0a8af2dae352cf56585f205b3b0f9c6c93b15b2b: Status 404 returned error can't find the container with id 7f0b602b2e26453dce9d4b6b0a8af2dae352cf56585f205b3b0f9c6c93b15b2b Jul 16 04:49:44.991581 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:44.991546 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" event={"ID":"fe086561-4ba6-410b-a3af-2d0dd032f572","Type":"ContainerStarted","Data":"011a2dba1375776411388127ef90e52f8feb2fa98abab265b070f6e590b8abe3"} Jul 16 04:49:44.991949 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:44.991600 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" event={"ID":"fe086561-4ba6-410b-a3af-2d0dd032f572","Type":"ContainerStarted","Data":"7f0b602b2e26453dce9d4b6b0a8af2dae352cf56585f205b3b0f9c6c93b15b2b"} Jul 16 04:49:45.009910 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:45.009858 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" podStartSLOduration=1.7551220010000002 podStartE2EDuration="2.009842743s" totalImagesPullingTime="254.720742ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:49:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:49:45.008706918 +0000 UTC m=+637.734175451" watchObservedRunningTime="2026-07-16 04:49:45.009842743 +0000 UTC m=+637.735311593" Jul 16 04:49:45.722141 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:45.722105 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:49:45.823926 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:45.823892 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:49:52.938386 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:52.938346 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:49:54.107014 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:54.106983 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:54.107382 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:54.107033 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:54.119352 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:54.119325 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:49:55.050758 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:49:55.050725 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gqnlk" Jul 16 04:50:43.256961 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:43.256926 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56cfd66f6d-9pxxq"] Jul 16 04:50:43.260810 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:43.260787 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56cfd66f6d-9pxxq" Jul 16 04:50:43.268060 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:43.267823 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56cfd66f6d-9pxxq"] Jul 16 04:50:43.356727 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:43.356693 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c069473c-638f-472f-aba7-f32ed57adc6e-tls-cert\") pod \"authorino-56cfd66f6d-9pxxq\" (UID: \"c069473c-638f-472f-aba7-f32ed57adc6e\") " pod="kuadrant-system/authorino-56cfd66f6d-9pxxq" Jul 16 04:50:43.356880 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:43.356733 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f99fc\" (UniqueName: \"kubernetes.io/projected/c069473c-638f-472f-aba7-f32ed57adc6e-kube-api-access-f99fc\") pod \"authorino-56cfd66f6d-9pxxq\" (UID: \"c069473c-638f-472f-aba7-f32ed57adc6e\") " pod="kuadrant-system/authorino-56cfd66f6d-9pxxq" Jul 16 04:50:43.457792 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:43.457764 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c069473c-638f-472f-aba7-f32ed57adc6e-tls-cert\") pod \"authorino-56cfd66f6d-9pxxq\" (UID: \"c069473c-638f-472f-aba7-f32ed57adc6e\") " pod="kuadrant-system/authorino-56cfd66f6d-9pxxq" Jul 16 04:50:43.457975 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:43.457806 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f99fc\" (UniqueName: \"kubernetes.io/projected/c069473c-638f-472f-aba7-f32ed57adc6e-kube-api-access-f99fc\") pod \"authorino-56cfd66f6d-9pxxq\" (UID: \"c069473c-638f-472f-aba7-f32ed57adc6e\") " pod="kuadrant-system/authorino-56cfd66f6d-9pxxq" Jul 16 04:50:43.460154 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:43.460136 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c069473c-638f-472f-aba7-f32ed57adc6e-tls-cert\") pod \"authorino-56cfd66f6d-9pxxq\" (UID: \"c069473c-638f-472f-aba7-f32ed57adc6e\") " pod="kuadrant-system/authorino-56cfd66f6d-9pxxq" Jul 16 04:50:43.465996 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:43.465977 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f99fc\" (UniqueName: \"kubernetes.io/projected/c069473c-638f-472f-aba7-f32ed57adc6e-kube-api-access-f99fc\") pod \"authorino-56cfd66f6d-9pxxq\" (UID: \"c069473c-638f-472f-aba7-f32ed57adc6e\") " pod="kuadrant-system/authorino-56cfd66f6d-9pxxq" Jul 16 04:50:43.556307 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:43.556232 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:50:43.571124 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:43.571100 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56cfd66f6d-9pxxq" Jul 16 04:50:43.656971 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:43.656881 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:50:43.702464 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:43.702389 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56cfd66f6d-9pxxq"] Jul 16 04:50:43.704959 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:50:43.704930 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc069473c_638f_472f_aba7_f32ed57adc6e.slice/crio-210eab6985298d9ff72dff4c7f953ff744d0fd77004c00e8c2db6be9187bfca2 WatchSource:0}: Error finding container 210eab6985298d9ff72dff4c7f953ff744d0fd77004c00e8c2db6be9187bfca2: Status 404 returned error can't find the container with id 210eab6985298d9ff72dff4c7f953ff744d0fd77004c00e8c2db6be9187bfca2 Jul 16 04:50:43.792583 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:43.792544 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:50:44.281577 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:44.281534 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56cfd66f6d-9pxxq" event={"ID":"c069473c-638f-472f-aba7-f32ed57adc6e","Type":"ContainerStarted","Data":"201e317dac102335d6550d63dc9780a8085b4339b2c7f36768d1db17c53de33a"} Jul 16 04:50:44.281577 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:44.281582 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56cfd66f6d-9pxxq" event={"ID":"c069473c-638f-472f-aba7-f32ed57adc6e","Type":"ContainerStarted","Data":"210eab6985298d9ff72dff4c7f953ff744d0fd77004c00e8c2db6be9187bfca2"} Jul 16 04:50:44.314166 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:44.314103 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-56cfd66f6d-9pxxq" podStartSLOduration=0.868043364 podStartE2EDuration="1.314087905s" totalImagesPullingTime="446.044541ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:50:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:44.312442007 +0000 UTC m=+697.037910554" watchObservedRunningTime="2026-07-16 04:50:44.314087905 +0000 UTC m=+697.039556427" Jul 16 04:50:44.364396 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:44.364360 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-64c99c69b5-6cf8c"] Jul 16 04:50:44.364617 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:44.364576 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-64c99c69b5-6cf8c" podUID="225461bb-ee5a-44f5-ae9e-04c612557e9d" containerName="authorino" containerID="cri-o://38a20803ab736b6375b9621c65fd8625c09d981a931663ba6243bd7d858deadc" gracePeriod=30 Jul 16 04:50:44.634987 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:44.634955 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64c99c69b5-6cf8c" Jul 16 04:50:44.669094 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:44.669063 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/225461bb-ee5a-44f5-ae9e-04c612557e9d-tls-cert\") pod \"225461bb-ee5a-44f5-ae9e-04c612557e9d\" (UID: \"225461bb-ee5a-44f5-ae9e-04c612557e9d\") " Jul 16 04:50:44.669308 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:44.669151 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fttsc\" (UniqueName: \"kubernetes.io/projected/225461bb-ee5a-44f5-ae9e-04c612557e9d-kube-api-access-fttsc\") pod \"225461bb-ee5a-44f5-ae9e-04c612557e9d\" (UID: \"225461bb-ee5a-44f5-ae9e-04c612557e9d\") " Jul 16 04:50:44.671892 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:44.671851 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/225461bb-ee5a-44f5-ae9e-04c612557e9d-kube-api-access-fttsc" (OuterVolumeSpecName: "kube-api-access-fttsc") pod "225461bb-ee5a-44f5-ae9e-04c612557e9d" (UID: "225461bb-ee5a-44f5-ae9e-04c612557e9d"). InnerVolumeSpecName "kube-api-access-fttsc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:50:44.694948 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:44.694911 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/225461bb-ee5a-44f5-ae9e-04c612557e9d-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "225461bb-ee5a-44f5-ae9e-04c612557e9d" (UID: "225461bb-ee5a-44f5-ae9e-04c612557e9d"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:50:44.770602 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:44.770515 2581 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/225461bb-ee5a-44f5-ae9e-04c612557e9d-tls-cert\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:50:44.770602 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:44.770553 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fttsc\" (UniqueName: \"kubernetes.io/projected/225461bb-ee5a-44f5-ae9e-04c612557e9d-kube-api-access-fttsc\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 04:50:44.772748 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:44.772715 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:50:44.881381 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:44.881349 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:50:45.290579 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:45.290543 2581 generic.go:356] "Generic (PLEG): container finished" podID="225461bb-ee5a-44f5-ae9e-04c612557e9d" containerID="38a20803ab736b6375b9621c65fd8625c09d981a931663ba6243bd7d858deadc" exitCode=0 Jul 16 04:50:45.291051 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:45.290632 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64c99c69b5-6cf8c" event={"ID":"225461bb-ee5a-44f5-ae9e-04c612557e9d","Type":"ContainerDied","Data":"38a20803ab736b6375b9621c65fd8625c09d981a931663ba6243bd7d858deadc"} Jul 16 04:50:45.291051 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:45.290692 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64c99c69b5-6cf8c" event={"ID":"225461bb-ee5a-44f5-ae9e-04c612557e9d","Type":"ContainerDied","Data":"deb6c2e5eff15a1219a3870fb11e64d96c8e16129514c8c9c2c36b3e1397b562"} Jul 16 04:50:45.291051 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:45.290716 2581 scope.go:117] "RemoveContainer" containerID="38a20803ab736b6375b9621c65fd8625c09d981a931663ba6243bd7d858deadc" Jul 16 04:50:45.291265 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:45.291247 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64c99c69b5-6cf8c" Jul 16 04:50:45.302864 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:45.302837 2581 scope.go:117] "RemoveContainer" containerID="38a20803ab736b6375b9621c65fd8625c09d981a931663ba6243bd7d858deadc" Jul 16 04:50:45.303144 ip-10-0-132-245 kubenswrapper[2581]: E0716 04:50:45.303124 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38a20803ab736b6375b9621c65fd8625c09d981a931663ba6243bd7d858deadc\": container with ID starting with 38a20803ab736b6375b9621c65fd8625c09d981a931663ba6243bd7d858deadc not found: ID does not exist" containerID="38a20803ab736b6375b9621c65fd8625c09d981a931663ba6243bd7d858deadc" Jul 16 04:50:45.303223 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:45.303154 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38a20803ab736b6375b9621c65fd8625c09d981a931663ba6243bd7d858deadc"} err="failed to get container status \"38a20803ab736b6375b9621c65fd8625c09d981a931663ba6243bd7d858deadc\": rpc error: code = NotFound desc = could not find container \"38a20803ab736b6375b9621c65fd8625c09d981a931663ba6243bd7d858deadc\": container with ID starting with 38a20803ab736b6375b9621c65fd8625c09d981a931663ba6243bd7d858deadc not found: ID does not exist" Jul 16 04:50:45.318933 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:45.318901 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-64c99c69b5-6cf8c"] Jul 16 04:50:45.325964 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:45.325934 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-64c99c69b5-6cf8c"] Jul 16 04:50:45.843312 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:45.843269 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="225461bb-ee5a-44f5-ae9e-04c612557e9d" path="/var/lib/kubelet/pods/225461bb-ee5a-44f5-ae9e-04c612557e9d/volumes" Jul 16 04:50:47.720741 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:50:47.720701 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:51:43.382066 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:51:43.382033 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:52:15.524316 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:15.524276 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:52:17.224388 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:17.224358 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:52:17.335982 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:17.335952 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b6d84dc7c-kljmf"] Jul 16 04:52:17.336350 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:17.336339 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="225461bb-ee5a-44f5-ae9e-04c612557e9d" containerName="authorino" Jul 16 04:52:17.336393 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:17.336353 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="225461bb-ee5a-44f5-ae9e-04c612557e9d" containerName="authorino" Jul 16 04:52:17.336426 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:17.336416 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="225461bb-ee5a-44f5-ae9e-04c612557e9d" containerName="authorino" Jul 16 04:52:17.339420 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:17.339404 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b6d84dc7c-kljmf" Jul 16 04:52:17.342992 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:17.342966 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6ds6k\"" Jul 16 04:52:17.342992 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:17.342991 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 16 04:52:17.343165 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:17.342970 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 16 04:52:17.345573 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:17.345548 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b6d84dc7c-kljmf"] Jul 16 04:52:17.392490 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:17.392457 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rflkf\" (UniqueName: \"kubernetes.io/projected/b261b326-c0c6-4bb6-8122-3a91bb8301ab-kube-api-access-rflkf\") pod \"maas-controller-5b6d84dc7c-kljmf\" (UID: \"b261b326-c0c6-4bb6-8122-3a91bb8301ab\") " pod="opendatahub/maas-controller-5b6d84dc7c-kljmf" Jul 16 04:52:17.392659 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:17.392547 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b261b326-c0c6-4bb6-8122-3a91bb8301ab-webhook-cert\") pod \"maas-controller-5b6d84dc7c-kljmf\" (UID: \"b261b326-c0c6-4bb6-8122-3a91bb8301ab\") " pod="opendatahub/maas-controller-5b6d84dc7c-kljmf" Jul 16 04:52:17.493433 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:17.493358 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b261b326-c0c6-4bb6-8122-3a91bb8301ab-webhook-cert\") pod \"maas-controller-5b6d84dc7c-kljmf\" (UID: \"b261b326-c0c6-4bb6-8122-3a91bb8301ab\") " pod="opendatahub/maas-controller-5b6d84dc7c-kljmf" Jul 16 04:52:17.493600 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:17.493450 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rflkf\" (UniqueName: \"kubernetes.io/projected/b261b326-c0c6-4bb6-8122-3a91bb8301ab-kube-api-access-rflkf\") pod \"maas-controller-5b6d84dc7c-kljmf\" (UID: \"b261b326-c0c6-4bb6-8122-3a91bb8301ab\") " pod="opendatahub/maas-controller-5b6d84dc7c-kljmf" Jul 16 04:52:17.495787 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:17.495763 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b261b326-c0c6-4bb6-8122-3a91bb8301ab-webhook-cert\") pod \"maas-controller-5b6d84dc7c-kljmf\" (UID: \"b261b326-c0c6-4bb6-8122-3a91bb8301ab\") " pod="opendatahub/maas-controller-5b6d84dc7c-kljmf" Jul 16 04:52:17.502137 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:17.502110 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rflkf\" (UniqueName: \"kubernetes.io/projected/b261b326-c0c6-4bb6-8122-3a91bb8301ab-kube-api-access-rflkf\") pod \"maas-controller-5b6d84dc7c-kljmf\" (UID: \"b261b326-c0c6-4bb6-8122-3a91bb8301ab\") " pod="opendatahub/maas-controller-5b6d84dc7c-kljmf" Jul 16 04:52:17.650806 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:17.650761 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b6d84dc7c-kljmf" Jul 16 04:52:17.777396 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:17.777373 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b6d84dc7c-kljmf"] Jul 16 04:52:17.779361 ip-10-0-132-245 kubenswrapper[2581]: W0716 04:52:17.779326 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb261b326_c0c6_4bb6_8122_3a91bb8301ab.slice/crio-316df52dcc8aa1685639bb627ee4dede137788d4d5528bf0d344df74c026db51 WatchSource:0}: Error finding container 316df52dcc8aa1685639bb627ee4dede137788d4d5528bf0d344df74c026db51: Status 404 returned error can't find the container with id 316df52dcc8aa1685639bb627ee4dede137788d4d5528bf0d344df74c026db51 Jul 16 04:52:18.739780 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:18.739746 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b6d84dc7c-kljmf" event={"ID":"b261b326-c0c6-4bb6-8122-3a91bb8301ab","Type":"ContainerStarted","Data":"c89f7c8b03463434e3b35a45b1ba038c1373ee3c40f3a30c3f22bd6ebd4abeb7"} Jul 16 04:52:18.740158 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:18.739805 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b6d84dc7c-kljmf" event={"ID":"b261b326-c0c6-4bb6-8122-3a91bb8301ab","Type":"ContainerStarted","Data":"316df52dcc8aa1685639bb627ee4dede137788d4d5528bf0d344df74c026db51"} Jul 16 04:52:18.740158 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:18.739832 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b6d84dc7c-kljmf" Jul 16 04:52:18.757410 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:18.757365 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b6d84dc7c-kljmf" podStartSLOduration=1.36311738 podStartE2EDuration="1.757352448s" totalImagesPullingTime="394.235068ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:52:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:52:18.755103153 +0000 UTC m=+791.480571689" watchObservedRunningTime="2026-07-16 04:52:18.757352448 +0000 UTC m=+791.482820998" Jul 16 04:52:29.747122 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:52:29.747091 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b6d84dc7c-kljmf" Jul 16 04:53:02.378905 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:53:02.378869 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:53:02.480736 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:53:02.480702 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:53:02.626096 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:53:02.626066 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:53:08.326272 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:53:08.326232 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:53:08.434871 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:53:08.434830 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:53:08.526254 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:53:08.526221 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:53:08.742324 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:53:08.742292 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:53:14.582498 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:53:14.582460 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:53:14.645914 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:53:14.645879 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:54:07.771080 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:54:07.771004 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrbbv_95a3a732-50cf-47f5-9629-0b3d241d017a/ovn-acl-logging/0.log" Jul 16 04:54:07.773293 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:54:07.772347 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrbbv_95a3a732-50cf-47f5-9629-0b3d241d017a/ovn-acl-logging/0.log" Jul 16 04:55:53.891994 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:55:53.891958 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:58:10.047934 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:58:10.047896 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:58:16.359870 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:58:16.359832 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:58:16.460146 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:58:16.460118 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 04:59:07.806163 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:59:07.806053 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrbbv_95a3a732-50cf-47f5-9629-0b3d241d017a/ovn-acl-logging/0.log" Jul 16 04:59:07.810283 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:59:07.809266 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrbbv_95a3a732-50cf-47f5-9629-0b3d241d017a/ovn-acl-logging/0.log" Jul 16 04:59:15.758062 ip-10-0-132-245 kubenswrapper[2581]: I0716 04:59:15.758012 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:03:21.820170 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:03:21.820092 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:03:28.698336 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:03:28.698300 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:06.954356 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:06.954324 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:07.839198 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:07.839094 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrbbv_95a3a732-50cf-47f5-9629-0b3d241d017a/ovn-acl-logging/0.log" Jul 16 05:04:07.844714 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:07.844686 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrbbv_95a3a732-50cf-47f5-9629-0b3d241d017a/ovn-acl-logging/0.log" Jul 16 05:04:17.625762 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:17.625669 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:25.883275 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:25.883240 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:33.336463 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:33.336383 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:33.911577 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:33.911545 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:34.032688 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:34.032652 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:34.373590 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:34.373519 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:40.030986 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.030953 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:40.138165 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.138132 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:40.222122 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.222093 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw"] Jul 16 05:04:40.225214 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.225193 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.229219 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.229197 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9f78d938-openshift-default-dockercfg-nd9jr\"" Jul 16 05:04:40.256266 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.256227 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw"] Jul 16 05:04:40.259729 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.259708 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:40.324888 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.324804 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvcmv\" (UniqueName: \"kubernetes.io/projected/fa9eeb5d-9661-419f-b1da-c0da240af867-kube-api-access-jvcmv\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.324888 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.324854 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-credential-socket\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.325090 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.324911 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-podinfo\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.325090 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.324944 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa9eeb5d-9661-419f-b1da-c0da240af867-istiod-ca-cert\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.325090 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.324976 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-workload-certs\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.325090 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.325024 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-data\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.325090 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.325044 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-workload-socket\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.325090 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.325060 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-envoy\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.325322 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.325149 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-token\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.369923 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.369889 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:40.426229 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.426202 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-workload-certs\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.426406 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.426254 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-data\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.426406 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.426288 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-workload-socket\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.426406 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.426305 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-envoy\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.426406 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.426357 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-token\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.426406 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.426398 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvcmv\" (UniqueName: \"kubernetes.io/projected/fa9eeb5d-9661-419f-b1da-c0da240af867-kube-api-access-jvcmv\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.426709 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.426432 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-credential-socket\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.426709 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.426452 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-podinfo\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.426709 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.426475 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa9eeb5d-9661-419f-b1da-c0da240af867-istiod-ca-cert\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.426709 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.426631 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-data\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.426709 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.426662 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-workload-certs\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.426927 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.426891 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-credential-socket\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.427070 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.427050 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-workload-socket\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.427170 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.427152 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa9eeb5d-9661-419f-b1da-c0da240af867-istiod-ca-cert\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.428576 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.428548 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-envoy\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.428924 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.428907 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-podinfo\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.438616 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.438593 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-token\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.444989 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.444964 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvcmv\" (UniqueName: \"kubernetes.io/projected/fa9eeb5d-9661-419f-b1da-c0da240af867-kube-api-access-jvcmv\") pod \"e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.454090 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.454063 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db"] Jul 16 05:04:40.461534 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.459704 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" Jul 16 05:04:40.462681 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.462659 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9f78d938-serving-cert\"" Jul 16 05:04:40.462805 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.462784 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-47w6t\"" Jul 16 05:04:40.468011 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.467678 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:40.469635 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.469610 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db"] Jul 16 05:04:40.535669 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.535638 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:40.564375 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.564343 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:40.628626 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.628551 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/897bc8b0-f24f-4244-a467-353847f67cbd-openshift-service-ca\") pod \"maas-api-e2e-ait-9f78d938-649cf7648c-fb7db\" (UID: \"897bc8b0-f24f-4244-a467-353847f67cbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" Jul 16 05:04:40.628823 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.628630 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/897bc8b0-f24f-4244-a467-353847f67cbd-maas-api-tls\") pod \"maas-api-e2e-ait-9f78d938-649cf7648c-fb7db\" (UID: \"897bc8b0-f24f-4244-a467-353847f67cbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" Jul 16 05:04:40.628823 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.628666 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hd77\" (UniqueName: \"kubernetes.io/projected/897bc8b0-f24f-4244-a467-353847f67cbd-kube-api-access-9hd77\") pod \"maas-api-e2e-ait-9f78d938-649cf7648c-fb7db\" (UID: \"897bc8b0-f24f-4244-a467-353847f67cbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" Jul 16 05:04:40.666824 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.666794 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:40.679192 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.679171 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw"] Jul 16 05:04:40.680485 ip-10-0-132-245 kubenswrapper[2581]: W0716 05:04:40.680464 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa9eeb5d_9661_419f_b1da_c0da240af867.slice/crio-1bb2cc5d92cb686ab6a1c5a4867e4e0435673bbea0ca041b13e38ff6566442a9 WatchSource:0}: Error finding container 1bb2cc5d92cb686ab6a1c5a4867e4e0435673bbea0ca041b13e38ff6566442a9: Status 404 returned error can't find the container with id 1bb2cc5d92cb686ab6a1c5a4867e4e0435673bbea0ca041b13e38ff6566442a9 Jul 16 05:04:40.682459 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.682434 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:04:40.682528 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.682490 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:04:40.682570 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.682538 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:04:40.730167 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.730144 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/897bc8b0-f24f-4244-a467-353847f67cbd-maas-api-tls\") pod \"maas-api-e2e-ait-9f78d938-649cf7648c-fb7db\" (UID: \"897bc8b0-f24f-4244-a467-353847f67cbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" Jul 16 05:04:40.730277 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.730177 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hd77\" (UniqueName: \"kubernetes.io/projected/897bc8b0-f24f-4244-a467-353847f67cbd-kube-api-access-9hd77\") pod \"maas-api-e2e-ait-9f78d938-649cf7648c-fb7db\" (UID: \"897bc8b0-f24f-4244-a467-353847f67cbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" Jul 16 05:04:40.730277 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.730272 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/897bc8b0-f24f-4244-a467-353847f67cbd-openshift-service-ca\") pod \"maas-api-e2e-ait-9f78d938-649cf7648c-fb7db\" (UID: \"897bc8b0-f24f-4244-a467-353847f67cbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" Jul 16 05:04:40.730883 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.730865 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/897bc8b0-f24f-4244-a467-353847f67cbd-openshift-service-ca\") pod \"maas-api-e2e-ait-9f78d938-649cf7648c-fb7db\" (UID: \"897bc8b0-f24f-4244-a467-353847f67cbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" Jul 16 05:04:40.732436 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.732413 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/897bc8b0-f24f-4244-a467-353847f67cbd-maas-api-tls\") pod \"maas-api-e2e-ait-9f78d938-649cf7648c-fb7db\" (UID: \"897bc8b0-f24f-4244-a467-353847f67cbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" Jul 16 05:04:40.738202 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.738183 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hd77\" (UniqueName: \"kubernetes.io/projected/897bc8b0-f24f-4244-a467-353847f67cbd-kube-api-access-9hd77\") pod \"maas-api-e2e-ait-9f78d938-649cf7648c-fb7db\" (UID: \"897bc8b0-f24f-4244-a467-353847f67cbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" Jul 16 05:04:40.778775 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.778746 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" Jul 16 05:04:40.926644 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.926619 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db"] Jul 16 05:04:40.928674 ip-10-0-132-245 kubenswrapper[2581]: W0716 05:04:40.928617 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod897bc8b0_f24f_4244_a467_353847f67cbd.slice/crio-d22ba8e51481a645ce6d5ce95ff4dd0afc12620d2213bdf86ab3378b8d04c6e5 WatchSource:0}: Error finding container d22ba8e51481a645ce6d5ce95ff4dd0afc12620d2213bdf86ab3378b8d04c6e5: Status 404 returned error can't find the container with id d22ba8e51481a645ce6d5ce95ff4dd0afc12620d2213bdf86ab3378b8d04c6e5 Jul 16 05:04:40.930344 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:40.930323 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 05:04:41.233907 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:41.233882 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:41.343971 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:41.343916 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:41.370427 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:41.370388 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" event={"ID":"897bc8b0-f24f-4244-a467-353847f67cbd","Type":"ContainerStarted","Data":"d22ba8e51481a645ce6d5ce95ff4dd0afc12620d2213bdf86ab3378b8d04c6e5"} Jul 16 05:04:41.372649 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:41.372623 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" event={"ID":"fa9eeb5d-9661-419f-b1da-c0da240af867","Type":"ContainerStarted","Data":"cfa4107200d272ad3d6e2ecf425ac14a651ddcc17661f94d5d3760fff6211035"} Jul 16 05:04:41.372767 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:41.372676 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" event={"ID":"fa9eeb5d-9661-419f-b1da-c0da240af867","Type":"ContainerStarted","Data":"1bb2cc5d92cb686ab6a1c5a4867e4e0435673bbea0ca041b13e38ff6566442a9"} Jul 16 05:04:41.395687 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:41.395621 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" podStartSLOduration=1.395597071 podStartE2EDuration="1.395597071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:04:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:04:41.394584927 +0000 UTC m=+1534.120053468" watchObservedRunningTime="2026-07-16 05:04:41.395597071 +0000 UTC m=+1534.121065594" Jul 16 05:04:41.536052 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:41.535964 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:41.541597 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:41.541573 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:42.379357 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:42.379318 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" event={"ID":"897bc8b0-f24f-4244-a467-353847f67cbd","Type":"ContainerStarted","Data":"3e7a6da1c5952a3a40d60e03223f4fa07161d1d6bbf0ae08e26d3d85610f75ad"} Jul 16 05:04:42.379785 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:42.379582 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:42.380379 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:42.380361 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:42.397729 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:42.397685 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" podStartSLOduration=1.6322314329999998 podStartE2EDuration="2.397672173s" totalImagesPullingTime="765.44074ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:04:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:04:42.396252715 +0000 UTC m=+1535.121721265" watchObservedRunningTime="2026-07-16 05:04:42.397672173 +0000 UTC m=+1535.123140696" Jul 16 05:04:43.383809 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:43.383774 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" Jul 16 05:04:44.377822 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:44.377783 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:46.515738 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:46.515701 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw"] Jul 16 05:04:46.516126 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:46.515875 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" podUID="fa9eeb5d-9661-419f-b1da-c0da240af867" containerName="istio-proxy" containerID="cri-o://cfa4107200d272ad3d6e2ecf425ac14a651ddcc17661f94d5d3760fff6211035" gracePeriod=30 Jul 16 05:04:46.544526 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:46.540717 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:46.654616 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:46.654578 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:48.127488 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:48.127457 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:49.394286 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:49.394259 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" Jul 16 05:04:51.968477 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:51.968450 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:52.034769 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.034701 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-data\") pod \"fa9eeb5d-9661-419f-b1da-c0da240af867\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " Jul 16 05:04:52.034769 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.034732 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-envoy\") pod \"fa9eeb5d-9661-419f-b1da-c0da240af867\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " Jul 16 05:04:52.034769 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.034759 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa9eeb5d-9661-419f-b1da-c0da240af867-istiod-ca-cert\") pod \"fa9eeb5d-9661-419f-b1da-c0da240af867\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " Jul 16 05:04:52.034968 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.034792 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-credential-socket\") pod \"fa9eeb5d-9661-419f-b1da-c0da240af867\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " Jul 16 05:04:52.034968 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.034822 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-token\") pod \"fa9eeb5d-9661-419f-b1da-c0da240af867\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " Jul 16 05:04:52.034968 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.034850 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvcmv\" (UniqueName: \"kubernetes.io/projected/fa9eeb5d-9661-419f-b1da-c0da240af867-kube-api-access-jvcmv\") pod \"fa9eeb5d-9661-419f-b1da-c0da240af867\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " Jul 16 05:04:52.034968 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.034888 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-workload-certs\") pod \"fa9eeb5d-9661-419f-b1da-c0da240af867\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " Jul 16 05:04:52.034968 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.034946 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-workload-socket\") pod \"fa9eeb5d-9661-419f-b1da-c0da240af867\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " Jul 16 05:04:52.034968 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.034968 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-podinfo\") pod \"fa9eeb5d-9661-419f-b1da-c0da240af867\" (UID: \"fa9eeb5d-9661-419f-b1da-c0da240af867\") " Jul 16 05:04:52.040006 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.039728 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fa9eeb5d-9661-419f-b1da-c0da240af867" (UID: "fa9eeb5d-9661-419f-b1da-c0da240af867"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:04:52.040006 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.039840 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fa9eeb5d-9661-419f-b1da-c0da240af867" (UID: "fa9eeb5d-9661-419f-b1da-c0da240af867"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:04:52.040006 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.039852 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa9eeb5d-9661-419f-b1da-c0da240af867-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fa9eeb5d-9661-419f-b1da-c0da240af867" (UID: "fa9eeb5d-9661-419f-b1da-c0da240af867"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:04:52.040006 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.039972 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fa9eeb5d-9661-419f-b1da-c0da240af867" (UID: "fa9eeb5d-9661-419f-b1da-c0da240af867"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:04:52.040320 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.040252 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-data" (OuterVolumeSpecName: "istio-data") pod "fa9eeb5d-9661-419f-b1da-c0da240af867" (UID: "fa9eeb5d-9661-419f-b1da-c0da240af867"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:04:52.041870 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.041841 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fa9eeb5d-9661-419f-b1da-c0da240af867" (UID: "fa9eeb5d-9661-419f-b1da-c0da240af867"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:04:52.041999 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.041897 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa9eeb5d-9661-419f-b1da-c0da240af867-kube-api-access-jvcmv" (OuterVolumeSpecName: "kube-api-access-jvcmv") pod "fa9eeb5d-9661-419f-b1da-c0da240af867" (UID: "fa9eeb5d-9661-419f-b1da-c0da240af867"). InnerVolumeSpecName "kube-api-access-jvcmv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:04:52.042108 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.042089 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-token" (OuterVolumeSpecName: "istio-token") pod "fa9eeb5d-9661-419f-b1da-c0da240af867" (UID: "fa9eeb5d-9661-419f-b1da-c0da240af867"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:04:52.042170 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.042142 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fa9eeb5d-9661-419f-b1da-c0da240af867" (UID: "fa9eeb5d-9661-419f-b1da-c0da240af867"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:04:52.135627 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.135598 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvcmv\" (UniqueName: \"kubernetes.io/projected/fa9eeb5d-9661-419f-b1da-c0da240af867-kube-api-access-jvcmv\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:04:52.135627 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.135623 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-workload-certs\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:04:52.135627 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.135634 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-workload-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:04:52.135828 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.135644 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-podinfo\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:04:52.135828 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.135653 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-data\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:04:52.135828 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.135661 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-envoy\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:04:52.135828 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.135670 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa9eeb5d-9661-419f-b1da-c0da240af867-istiod-ca-cert\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:04:52.135828 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.135677 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa9eeb5d-9661-419f-b1da-c0da240af867-credential-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:04:52.135828 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.135686 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa9eeb5d-9661-419f-b1da-c0da240af867-istio-token\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:04:52.173365 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.173337 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db"] Jul 16 05:04:52.173670 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.173605 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" podUID="897bc8b0-f24f-4244-a467-353847f67cbd" containerName="maas-api" containerID="cri-o://3e7a6da1c5952a3a40d60e03223f4fa07161d1d6bbf0ae08e26d3d85610f75ad" gracePeriod=30 Jul 16 05:04:52.202658 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.202634 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:52.410410 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.410388 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" Jul 16 05:04:52.427660 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.427632 2581 generic.go:356] "Generic (PLEG): container finished" podID="897bc8b0-f24f-4244-a467-353847f67cbd" containerID="3e7a6da1c5952a3a40d60e03223f4fa07161d1d6bbf0ae08e26d3d85610f75ad" exitCode=0 Jul 16 05:04:52.427776 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.427665 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" Jul 16 05:04:52.427776 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.427722 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" event={"ID":"897bc8b0-f24f-4244-a467-353847f67cbd","Type":"ContainerDied","Data":"3e7a6da1c5952a3a40d60e03223f4fa07161d1d6bbf0ae08e26d3d85610f75ad"} Jul 16 05:04:52.427776 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.427769 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db" event={"ID":"897bc8b0-f24f-4244-a467-353847f67cbd","Type":"ContainerDied","Data":"d22ba8e51481a645ce6d5ce95ff4dd0afc12620d2213bdf86ab3378b8d04c6e5"} Jul 16 05:04:52.427921 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.427794 2581 scope.go:117] "RemoveContainer" containerID="3e7a6da1c5952a3a40d60e03223f4fa07161d1d6bbf0ae08e26d3d85610f75ad" Jul 16 05:04:52.429918 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.429332 2581 generic.go:356] "Generic (PLEG): container finished" podID="fa9eeb5d-9661-419f-b1da-c0da240af867" containerID="cfa4107200d272ad3d6e2ecf425ac14a651ddcc17661f94d5d3760fff6211035" exitCode=0 Jul 16 05:04:52.429918 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.429384 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" Jul 16 05:04:52.429918 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.429443 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" event={"ID":"fa9eeb5d-9661-419f-b1da-c0da240af867","Type":"ContainerDied","Data":"cfa4107200d272ad3d6e2ecf425ac14a651ddcc17661f94d5d3760fff6211035"} Jul 16 05:04:52.429918 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.429489 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw" event={"ID":"fa9eeb5d-9661-419f-b1da-c0da240af867","Type":"ContainerDied","Data":"1bb2cc5d92cb686ab6a1c5a4867e4e0435673bbea0ca041b13e38ff6566442a9"} Jul 16 05:04:52.437700 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.437676 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/897bc8b0-f24f-4244-a467-353847f67cbd-openshift-service-ca\") pod \"897bc8b0-f24f-4244-a467-353847f67cbd\" (UID: \"897bc8b0-f24f-4244-a467-353847f67cbd\") " Jul 16 05:04:52.437806 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.437732 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/897bc8b0-f24f-4244-a467-353847f67cbd-maas-api-tls\") pod \"897bc8b0-f24f-4244-a467-353847f67cbd\" (UID: \"897bc8b0-f24f-4244-a467-353847f67cbd\") " Jul 16 05:04:52.437806 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.437795 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hd77\" (UniqueName: \"kubernetes.io/projected/897bc8b0-f24f-4244-a467-353847f67cbd-kube-api-access-9hd77\") pod \"897bc8b0-f24f-4244-a467-353847f67cbd\" (UID: \"897bc8b0-f24f-4244-a467-353847f67cbd\") " Jul 16 05:04:52.438362 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.438333 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/897bc8b0-f24f-4244-a467-353847f67cbd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "897bc8b0-f24f-4244-a467-353847f67cbd" (UID: "897bc8b0-f24f-4244-a467-353847f67cbd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:04:52.439285 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.439243 2581 scope.go:117] "RemoveContainer" containerID="3e7a6da1c5952a3a40d60e03223f4fa07161d1d6bbf0ae08e26d3d85610f75ad" Jul 16 05:04:52.439655 ip-10-0-132-245 kubenswrapper[2581]: E0716 05:04:52.439619 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e7a6da1c5952a3a40d60e03223f4fa07161d1d6bbf0ae08e26d3d85610f75ad\": container with ID starting with 3e7a6da1c5952a3a40d60e03223f4fa07161d1d6bbf0ae08e26d3d85610f75ad not found: ID does not exist" containerID="3e7a6da1c5952a3a40d60e03223f4fa07161d1d6bbf0ae08e26d3d85610f75ad" Jul 16 05:04:52.439793 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.439768 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e7a6da1c5952a3a40d60e03223f4fa07161d1d6bbf0ae08e26d3d85610f75ad"} err="failed to get container status \"3e7a6da1c5952a3a40d60e03223f4fa07161d1d6bbf0ae08e26d3d85610f75ad\": rpc error: code = NotFound desc = could not find container \"3e7a6da1c5952a3a40d60e03223f4fa07161d1d6bbf0ae08e26d3d85610f75ad\": container with ID starting with 3e7a6da1c5952a3a40d60e03223f4fa07161d1d6bbf0ae08e26d3d85610f75ad not found: ID does not exist" Jul 16 05:04:52.439912 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.439899 2581 scope.go:117] "RemoveContainer" containerID="cfa4107200d272ad3d6e2ecf425ac14a651ddcc17661f94d5d3760fff6211035" Jul 16 05:04:52.440093 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.440059 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/897bc8b0-f24f-4244-a467-353847f67cbd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "897bc8b0-f24f-4244-a467-353847f67cbd" (UID: "897bc8b0-f24f-4244-a467-353847f67cbd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 05:04:52.440193 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.440100 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/897bc8b0-f24f-4244-a467-353847f67cbd-kube-api-access-9hd77" (OuterVolumeSpecName: "kube-api-access-9hd77") pod "897bc8b0-f24f-4244-a467-353847f67cbd" (UID: "897bc8b0-f24f-4244-a467-353847f67cbd"). InnerVolumeSpecName "kube-api-access-9hd77". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:04:52.448107 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.448090 2581 scope.go:117] "RemoveContainer" containerID="cfa4107200d272ad3d6e2ecf425ac14a651ddcc17661f94d5d3760fff6211035" Jul 16 05:04:52.448339 ip-10-0-132-245 kubenswrapper[2581]: E0716 05:04:52.448319 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfa4107200d272ad3d6e2ecf425ac14a651ddcc17661f94d5d3760fff6211035\": container with ID starting with cfa4107200d272ad3d6e2ecf425ac14a651ddcc17661f94d5d3760fff6211035 not found: ID does not exist" containerID="cfa4107200d272ad3d6e2ecf425ac14a651ddcc17661f94d5d3760fff6211035" Jul 16 05:04:52.448398 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.448346 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfa4107200d272ad3d6e2ecf425ac14a651ddcc17661f94d5d3760fff6211035"} err="failed to get container status \"cfa4107200d272ad3d6e2ecf425ac14a651ddcc17661f94d5d3760fff6211035\": rpc error: code = NotFound desc = could not find container \"cfa4107200d272ad3d6e2ecf425ac14a651ddcc17661f94d5d3760fff6211035\": container with ID starting with cfa4107200d272ad3d6e2ecf425ac14a651ddcc17661f94d5d3760fff6211035 not found: ID does not exist" Jul 16 05:04:52.455381 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.455362 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw"] Jul 16 05:04:52.461053 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.461033 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9f78d938-openshift-default-78bddcd99f-6rbbw"] Jul 16 05:04:52.539081 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.539056 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/897bc8b0-f24f-4244-a467-353847f67cbd-openshift-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:04:52.539081 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.539077 2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/897bc8b0-f24f-4244-a467-353847f67cbd-maas-api-tls\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:04:52.539249 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.539088 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hd77\" (UniqueName: \"kubernetes.io/projected/897bc8b0-f24f-4244-a467-353847f67cbd-kube-api-access-9hd77\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:04:52.554921 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.554894 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:52.749623 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.749599 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db"] Jul 16 05:04:52.753416 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:52.753387 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9f78d938-649cf7648c-fb7db"] Jul 16 05:04:53.839516 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:53.839472 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="897bc8b0-f24f-4244-a467-353847f67cbd" path="/var/lib/kubelet/pods/897bc8b0-f24f-4244-a467-353847f67cbd/volumes" Jul 16 05:04:53.839960 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:53.839944 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa9eeb5d-9661-419f-b1da-c0da240af867" path="/var/lib/kubelet/pods/fa9eeb5d-9661-419f-b1da-c0da240af867/volumes" Jul 16 05:04:58.083888 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.083852 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:58.184026 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.183997 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:58.358860 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.358763 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:58.450656 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.450620 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:58.517056 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.517025 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr"] Jul 16 05:04:58.517442 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.517430 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="897bc8b0-f24f-4244-a467-353847f67cbd" containerName="maas-api" Jul 16 05:04:58.517489 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.517443 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="897bc8b0-f24f-4244-a467-353847f67cbd" containerName="maas-api" Jul 16 05:04:58.517489 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.517458 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa9eeb5d-9661-419f-b1da-c0da240af867" containerName="istio-proxy" Jul 16 05:04:58.517489 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.517465 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa9eeb5d-9661-419f-b1da-c0da240af867" containerName="istio-proxy" Jul 16 05:04:58.517610 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.517537 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="897bc8b0-f24f-4244-a467-353847f67cbd" containerName="maas-api" Jul 16 05:04:58.517610 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.517548 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa9eeb5d-9661-419f-b1da-c0da240af867" containerName="istio-proxy" Jul 16 05:04:58.520829 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.520804 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.523196 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.523177 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-f6882454-gw-openshift-default-dockercfg-rwb5d\"" Jul 16 05:04:58.533071 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.533050 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr"] Jul 16 05:04:58.552243 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.552202 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:58.591173 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.591143 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-data\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.591274 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.591188 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-token\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.591274 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.591236 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-workload-certs\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.591274 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.591256 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-podinfo\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.591380 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.591290 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7pch\" (UniqueName: \"kubernetes.io/projected/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-kube-api-access-r7pch\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.591380 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.591321 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-envoy\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.591380 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.591342 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istiod-ca-cert\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.591380 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.591374 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-credential-socket\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.591499 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.591396 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-workload-socket\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.612984 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.612913 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:58.692379 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.692345 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-workload-certs\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.692581 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.692384 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-podinfo\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.692581 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.692420 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7pch\" (UniqueName: \"kubernetes.io/projected/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-kube-api-access-r7pch\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.692581 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.692446 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-envoy\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.692581 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.692472 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istiod-ca-cert\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.692581 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.692529 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-credential-socket\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.692581 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.692567 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-workload-socket\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.692910 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.692599 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-data\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.692910 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.692651 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-token\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.692910 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.692809 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-workload-certs\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.693072 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.692914 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-credential-socket\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.693072 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.693038 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-data\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.693175 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.693122 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-workload-socket\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.693293 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.693274 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istiod-ca-cert\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.695325 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.695294 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-envoy\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.695425 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.695404 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-podinfo\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.702621 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.702596 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-token\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.703383 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.703365 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7pch\" (UniqueName: \"kubernetes.io/projected/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-kube-api-access-r7pch\") pod \"e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.831700 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.831670 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:04:58.929431 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.929396 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:58.989451 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.989420 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr"] Jul 16 05:04:58.991385 ip-10-0-132-245 kubenswrapper[2581]: W0716 05:04:58.991360 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod869f99c7_b0ef_49f1_ba53_c1cc103e2d98.slice/crio-91308f2a19c2ea8da365d09905e95feabd196d22ee2a52a31254ffbacff84b28 WatchSource:0}: Error finding container 91308f2a19c2ea8da365d09905e95feabd196d22ee2a52a31254ffbacff84b28: Status 404 returned error can't find the container with id 91308f2a19c2ea8da365d09905e95feabd196d22ee2a52a31254ffbacff84b28 Jul 16 05:04:58.993495 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.993463 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:04:58.993592 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.993548 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:04:58.993592 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:58.993580 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:04:59.088689 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:59.088647 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:59.376626 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:59.376591 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:59.468873 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:59.468819 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" event={"ID":"869f99c7-b0ef-49f1-ba53-c1cc103e2d98","Type":"ContainerStarted","Data":"48f9f106c3c655747402fd5704bc144f768bf4450772ecd4123e0d2570fac55b"} Jul 16 05:04:59.469071 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:59.468894 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" event={"ID":"869f99c7-b0ef-49f1-ba53-c1cc103e2d98","Type":"ContainerStarted","Data":"91308f2a19c2ea8da365d09905e95feabd196d22ee2a52a31254ffbacff84b28"} Jul 16 05:04:59.490490 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:59.490426 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" podStartSLOduration=1.490406472 podStartE2EDuration="1.490406472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:04:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:04:59.488211425 +0000 UTC m=+1552.213679962" watchObservedRunningTime="2026-07-16 05:04:59.490406472 +0000 UTC m=+1552.215874995" Jul 16 05:04:59.549602 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:59.549573 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:59.654998 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:59.654919 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:04:59.831958 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:04:59.831931 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:05:00.837386 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:00.837350 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:05:01.478086 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:01.478053 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:05:01.478866 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:01.478845 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:05:02.120320 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:02.120280 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:04.390529 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:04.390485 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr"] Jul 16 05:05:04.419242 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:04.419215 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:04.491625 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:04.491585 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" podUID="869f99c7-b0ef-49f1-ba53-c1cc103e2d98" containerName="istio-proxy" containerID="cri-o://48f9f106c3c655747402fd5704bc144f768bf4450772ecd4123e0d2570fac55b" gracePeriod=30 Jul 16 05:05:04.539563 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:04.539397 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:06.369378 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:06.369336 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:10.263224 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.263195 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:10.474376 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.474350 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:10.525488 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.525387 2581 generic.go:356] "Generic (PLEG): container finished" podID="869f99c7-b0ef-49f1-ba53-c1cc103e2d98" containerID="48f9f106c3c655747402fd5704bc144f768bf4450772ecd4123e0d2570fac55b" exitCode=0 Jul 16 05:05:10.525488 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.525464 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" event={"ID":"869f99c7-b0ef-49f1-ba53-c1cc103e2d98","Type":"ContainerDied","Data":"48f9f106c3c655747402fd5704bc144f768bf4450772ecd4123e0d2570fac55b"} Jul 16 05:05:10.554311 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.554289 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:05:10.608782 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.608756 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-envoy\") pod \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " Jul 16 05:05:10.608967 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.608825 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-credential-socket\") pod \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " Jul 16 05:05:10.608967 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.608851 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istiod-ca-cert\") pod \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " Jul 16 05:05:10.608967 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.608871 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-token\") pod \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " Jul 16 05:05:10.608967 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.608915 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-workload-socket\") pod \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " Jul 16 05:05:10.608967 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.608931 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7pch\" (UniqueName: \"kubernetes.io/projected/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-kube-api-access-r7pch\") pod \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " Jul 16 05:05:10.608967 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.608961 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-podinfo\") pod \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " Jul 16 05:05:10.609293 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.608991 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-data\") pod \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " Jul 16 05:05:10.609293 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.609009 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-workload-certs\") pod \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\" (UID: \"869f99c7-b0ef-49f1-ba53-c1cc103e2d98\") " Jul 16 05:05:10.609293 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.609187 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "869f99c7-b0ef-49f1-ba53-c1cc103e2d98" (UID: "869f99c7-b0ef-49f1-ba53-c1cc103e2d98"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:05:10.609293 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.609199 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "869f99c7-b0ef-49f1-ba53-c1cc103e2d98" (UID: "869f99c7-b0ef-49f1-ba53-c1cc103e2d98"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:05:10.609293 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.609266 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "869f99c7-b0ef-49f1-ba53-c1cc103e2d98" (UID: "869f99c7-b0ef-49f1-ba53-c1cc103e2d98"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:05:10.609592 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.609321 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istiod-ca-cert\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:05:10.609592 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.609341 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-workload-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:05:10.609592 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.609355 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-credential-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:05:10.609592 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.609533 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "869f99c7-b0ef-49f1-ba53-c1cc103e2d98" (UID: "869f99c7-b0ef-49f1-ba53-c1cc103e2d98"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:05:10.609960 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.609941 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-data" (OuterVolumeSpecName: "istio-data") pod "869f99c7-b0ef-49f1-ba53-c1cc103e2d98" (UID: "869f99c7-b0ef-49f1-ba53-c1cc103e2d98"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:05:10.611067 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.611045 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "869f99c7-b0ef-49f1-ba53-c1cc103e2d98" (UID: "869f99c7-b0ef-49f1-ba53-c1cc103e2d98"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:05:10.611166 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.611133 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "869f99c7-b0ef-49f1-ba53-c1cc103e2d98" (UID: "869f99c7-b0ef-49f1-ba53-c1cc103e2d98"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:05:10.611450 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.611427 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-token" (OuterVolumeSpecName: "istio-token") pod "869f99c7-b0ef-49f1-ba53-c1cc103e2d98" (UID: "869f99c7-b0ef-49f1-ba53-c1cc103e2d98"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:05:10.611564 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.611455 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-kube-api-access-r7pch" (OuterVolumeSpecName: "kube-api-access-r7pch") pod "869f99c7-b0ef-49f1-ba53-c1cc103e2d98" (UID: "869f99c7-b0ef-49f1-ba53-c1cc103e2d98"). InnerVolumeSpecName "kube-api-access-r7pch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:05:10.710174 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.710141 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-token\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:05:10.710174 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.710171 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7pch\" (UniqueName: \"kubernetes.io/projected/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-kube-api-access-r7pch\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:05:10.710174 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.710182 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-podinfo\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:05:10.710426 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.710190 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-data\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:05:10.710426 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.710199 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-workload-certs\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:05:10.710426 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:10.710207 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/869f99c7-b0ef-49f1-ba53-c1cc103e2d98-istio-envoy\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:05:11.531746 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:11.531712 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" Jul 16 05:05:11.531746 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:11.531737 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr" event={"ID":"869f99c7-b0ef-49f1-ba53-c1cc103e2d98","Type":"ContainerDied","Data":"91308f2a19c2ea8da365d09905e95feabd196d22ee2a52a31254ffbacff84b28"} Jul 16 05:05:11.532204 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:11.531802 2581 scope.go:117] "RemoveContainer" containerID="48f9f106c3c655747402fd5704bc144f768bf4450772ecd4123e0d2570fac55b" Jul 16 05:05:11.553744 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:11.553718 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr"] Jul 16 05:05:11.557143 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:11.557122 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-f6882454-gw-openshift-default-654dfdb567-nptzr"] Jul 16 05:05:11.839646 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:11.839565 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="869f99c7-b0ef-49f1-ba53-c1cc103e2d98" path="/var/lib/kubelet/pods/869f99c7-b0ef-49f1-ba53-c1cc103e2d98/volumes" Jul 16 05:05:16.178996 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.178958 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:16.283855 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.283809 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:16.358921 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.358892 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98"] Jul 16 05:05:16.359395 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.359375 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="869f99c7-b0ef-49f1-ba53-c1cc103e2d98" containerName="istio-proxy" Jul 16 05:05:16.359395 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.359395 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="869f99c7-b0ef-49f1-ba53-c1cc103e2d98" containerName="istio-proxy" Jul 16 05:05:16.359591 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.359528 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="869f99c7-b0ef-49f1-ba53-c1cc103e2d98" containerName="istio-proxy" Jul 16 05:05:16.362943 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.362920 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.366083 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.366065 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9116b405-openshift-default-dockercfg-42mtl\"" Jul 16 05:05:16.380916 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.380888 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98"] Jul 16 05:05:16.411748 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.411726 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:16.460247 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.460220 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b95gc\" (UniqueName: \"kubernetes.io/projected/b6380b74-143a-4474-bf20-59c97fb3b639-kube-api-access-b95gc\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.460401 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.460263 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-istio-data\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.460401 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.460332 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6380b74-143a-4474-bf20-59c97fb3b639-istio-podinfo\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.460401 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.460365 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-workload-socket\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.460401 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.460393 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6380b74-143a-4474-bf20-59c97fb3b639-istio-token\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.460589 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.460419 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-workload-certs\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.460589 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.460462 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-istio-envoy\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.460589 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.460496 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-credential-socket\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.460589 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.460557 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6380b74-143a-4474-bf20-59c97fb3b639-istiod-ca-cert\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.496539 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.496499 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:16.561580 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.561539 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-istio-envoy\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.561748 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.561634 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-credential-socket\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.562099 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.562050 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-credential-socket\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.562260 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.562243 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6380b74-143a-4474-bf20-59c97fb3b639-istiod-ca-cert\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.562423 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.562407 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b95gc\" (UniqueName: \"kubernetes.io/projected/b6380b74-143a-4474-bf20-59c97fb3b639-kube-api-access-b95gc\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.562562 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.562547 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-istio-data\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.564639 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.562979 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6380b74-143a-4474-bf20-59c97fb3b639-istiod-ca-cert\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.564639 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.563026 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-istio-data\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.564639 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.562689 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6380b74-143a-4474-bf20-59c97fb3b639-istio-podinfo\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.564639 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.563982 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-workload-socket\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.564639 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.564012 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6380b74-143a-4474-bf20-59c97fb3b639-istio-token\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.564639 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.564042 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-workload-certs\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.564639 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.564398 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-workload-certs\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.564639 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.564581 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-workload-socket\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.564639 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.564637 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-istio-envoy\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.565563 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.565541 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6380b74-143a-4474-bf20-59c97fb3b639-istio-podinfo\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.577997 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.577970 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b95gc\" (UniqueName: \"kubernetes.io/projected/b6380b74-143a-4474-bf20-59c97fb3b639-kube-api-access-b95gc\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.578926 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.578906 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6380b74-143a-4474-bf20-59c97fb3b639-istio-token\") pod \"e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.579266 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.579240 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:16.652856 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.652831 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:16.675169 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.675140 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:16.860566 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.860538 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98"] Jul 16 05:05:16.861949 ip-10-0-132-245 kubenswrapper[2581]: W0716 05:05:16.861926 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6380b74_143a_4474_bf20_59c97fb3b639.slice/crio-3b741eb3f5e194f233614b05a7593b57e7480f2b7984ffc69deda8cb7007ea48 WatchSource:0}: Error finding container 3b741eb3f5e194f233614b05a7593b57e7480f2b7984ffc69deda8cb7007ea48: Status 404 returned error can't find the container with id 3b741eb3f5e194f233614b05a7593b57e7480f2b7984ffc69deda8cb7007ea48 Jul 16 05:05:16.864076 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.864040 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:05:16.864130 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.864111 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:05:16.864165 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.864137 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:05:16.992466 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:16.992387 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:17.181234 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:17.181202 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:17.283603 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:17.283515 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:17.572461 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:17.572381 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" event={"ID":"b6380b74-143a-4474-bf20-59c97fb3b639","Type":"ContainerStarted","Data":"731bf774f976698c97a4de904307cc21ace385e4777158110991e75dae1be554"} Jul 16 05:05:17.572461 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:17.572433 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" event={"ID":"b6380b74-143a-4474-bf20-59c97fb3b639","Type":"ContainerStarted","Data":"3b741eb3f5e194f233614b05a7593b57e7480f2b7984ffc69deda8cb7007ea48"} Jul 16 05:05:17.594637 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:17.594591 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" podStartSLOduration=1.594575326 podStartE2EDuration="1.594575326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:05:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:05:17.591908757 +0000 UTC m=+1570.317377293" watchObservedRunningTime="2026-07-16 05:05:17.594575326 +0000 UTC m=+1570.320043848" Jul 16 05:05:17.675645 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:17.675619 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:17.680185 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:17.680162 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:18.578494 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:18.578462 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:18.579471 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:18.579452 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:19.923458 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:19.923408 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:22.023015 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:22.022980 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98"] Jul 16 05:05:22.023455 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:22.023149 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" podUID="b6380b74-143a-4474-bf20-59c97fb3b639" containerName="istio-proxy" containerID="cri-o://731bf774f976698c97a4de904307cc21ace385e4777158110991e75dae1be554" gracePeriod=30 Jul 16 05:05:22.049226 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:22.049184 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:22.151220 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:22.151179 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:25.936491 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:25.936459 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:27.837077 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:27.837045 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:28.120150 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.120122 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:28.166762 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.166740 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:28.278157 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.278128 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-istio-envoy\") pod \"b6380b74-143a-4474-bf20-59c97fb3b639\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " Jul 16 05:05:28.278157 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.278163 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-workload-certs\") pod \"b6380b74-143a-4474-bf20-59c97fb3b639\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " Jul 16 05:05:28.278387 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.278185 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b95gc\" (UniqueName: \"kubernetes.io/projected/b6380b74-143a-4474-bf20-59c97fb3b639-kube-api-access-b95gc\") pod \"b6380b74-143a-4474-bf20-59c97fb3b639\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " Jul 16 05:05:28.278387 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.278210 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6380b74-143a-4474-bf20-59c97fb3b639-istiod-ca-cert\") pod \"b6380b74-143a-4474-bf20-59c97fb3b639\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " Jul 16 05:05:28.278387 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.278241 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-istio-data\") pod \"b6380b74-143a-4474-bf20-59c97fb3b639\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " Jul 16 05:05:28.278387 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.278287 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6380b74-143a-4474-bf20-59c97fb3b639-istio-token\") pod \"b6380b74-143a-4474-bf20-59c97fb3b639\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " Jul 16 05:05:28.278387 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.278303 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-credential-socket\") pod \"b6380b74-143a-4474-bf20-59c97fb3b639\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " Jul 16 05:05:28.278387 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.278328 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-workload-socket\") pod \"b6380b74-143a-4474-bf20-59c97fb3b639\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " Jul 16 05:05:28.278387 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.278387 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6380b74-143a-4474-bf20-59c97fb3b639-istio-podinfo\") pod \"b6380b74-143a-4474-bf20-59c97fb3b639\" (UID: \"b6380b74-143a-4474-bf20-59c97fb3b639\") " Jul 16 05:05:28.280673 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.280645 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b6380b74-143a-4474-bf20-59c97fb3b639" (UID: "b6380b74-143a-4474-bf20-59c97fb3b639"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:05:28.280971 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.280941 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b6380b74-143a-4474-bf20-59c97fb3b639" (UID: "b6380b74-143a-4474-bf20-59c97fb3b639"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:05:28.281083 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.281042 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b6380b74-143a-4474-bf20-59c97fb3b639" (UID: "b6380b74-143a-4474-bf20-59c97fb3b639"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:05:28.281083 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.281066 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6380b74-143a-4474-bf20-59c97fb3b639-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b6380b74-143a-4474-bf20-59c97fb3b639" (UID: "b6380b74-143a-4474-bf20-59c97fb3b639"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:05:28.281536 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.281492 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-istio-data" (OuterVolumeSpecName: "istio-data") pod "b6380b74-143a-4474-bf20-59c97fb3b639" (UID: "b6380b74-143a-4474-bf20-59c97fb3b639"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:05:28.282775 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.282744 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6380b74-143a-4474-bf20-59c97fb3b639-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b6380b74-143a-4474-bf20-59c97fb3b639" (UID: "b6380b74-143a-4474-bf20-59c97fb3b639"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:05:28.282860 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.282748 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6380b74-143a-4474-bf20-59c97fb3b639-kube-api-access-b95gc" (OuterVolumeSpecName: "kube-api-access-b95gc") pod "b6380b74-143a-4474-bf20-59c97fb3b639" (UID: "b6380b74-143a-4474-bf20-59c97fb3b639"). InnerVolumeSpecName "kube-api-access-b95gc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:05:28.282860 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.282752 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b6380b74-143a-4474-bf20-59c97fb3b639" (UID: "b6380b74-143a-4474-bf20-59c97fb3b639"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:05:28.282977 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.282959 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6380b74-143a-4474-bf20-59c97fb3b639-istio-token" (OuterVolumeSpecName: "istio-token") pod "b6380b74-143a-4474-bf20-59c97fb3b639" (UID: "b6380b74-143a-4474-bf20-59c97fb3b639"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:05:28.379441 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.379355 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-istio-envoy\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:05:28.379441 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.379385 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-workload-certs\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:05:28.379441 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.379395 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b95gc\" (UniqueName: \"kubernetes.io/projected/b6380b74-143a-4474-bf20-59c97fb3b639-kube-api-access-b95gc\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:05:28.379441 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.379407 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6380b74-143a-4474-bf20-59c97fb3b639-istiod-ca-cert\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:05:28.379441 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.379417 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-istio-data\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:05:28.379441 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.379425 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6380b74-143a-4474-bf20-59c97fb3b639-istio-token\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:05:28.379441 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.379434 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-credential-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:05:28.379441 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.379447 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6380b74-143a-4474-bf20-59c97fb3b639-workload-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:05:28.379868 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.379457 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6380b74-143a-4474-bf20-59c97fb3b639-istio-podinfo\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:05:28.629755 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.629656 2581 generic.go:356] "Generic (PLEG): container finished" podID="b6380b74-143a-4474-bf20-59c97fb3b639" containerID="731bf774f976698c97a4de904307cc21ace385e4777158110991e75dae1be554" exitCode=0 Jul 16 05:05:28.629755 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.629706 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" Jul 16 05:05:28.629755 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.629716 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" event={"ID":"b6380b74-143a-4474-bf20-59c97fb3b639","Type":"ContainerDied","Data":"731bf774f976698c97a4de904307cc21ace385e4777158110991e75dae1be554"} Jul 16 05:05:28.629755 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.629757 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98" event={"ID":"b6380b74-143a-4474-bf20-59c97fb3b639","Type":"ContainerDied","Data":"3b741eb3f5e194f233614b05a7593b57e7480f2b7984ffc69deda8cb7007ea48"} Jul 16 05:05:28.630083 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.629779 2581 scope.go:117] "RemoveContainer" containerID="731bf774f976698c97a4de904307cc21ace385e4777158110991e75dae1be554" Jul 16 05:05:28.639130 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.639113 2581 scope.go:117] "RemoveContainer" containerID="731bf774f976698c97a4de904307cc21ace385e4777158110991e75dae1be554" Jul 16 05:05:28.639380 ip-10-0-132-245 kubenswrapper[2581]: E0716 05:05:28.639362 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"731bf774f976698c97a4de904307cc21ace385e4777158110991e75dae1be554\": container with ID starting with 731bf774f976698c97a4de904307cc21ace385e4777158110991e75dae1be554 not found: ID does not exist" containerID="731bf774f976698c97a4de904307cc21ace385e4777158110991e75dae1be554" Jul 16 05:05:28.639432 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.639387 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"731bf774f976698c97a4de904307cc21ace385e4777158110991e75dae1be554"} err="failed to get container status \"731bf774f976698c97a4de904307cc21ace385e4777158110991e75dae1be554\": rpc error: code = NotFound desc = could not find container \"731bf774f976698c97a4de904307cc21ace385e4777158110991e75dae1be554\": container with ID starting with 731bf774f976698c97a4de904307cc21ace385e4777158110991e75dae1be554 not found: ID does not exist" Jul 16 05:05:28.651867 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.651843 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98"] Jul 16 05:05:28.655618 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:28.655596 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9116b405-openshift-default-8fbdcb99f-6cm98"] Jul 16 05:05:29.839784 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:29.839743 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6380b74-143a-4474-bf20-59c97fb3b639" path="/var/lib/kubelet/pods/b6380b74-143a-4474-bf20-59c97fb3b639/volumes" Jul 16 05:05:33.713839 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:33.713804 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:33.774376 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:33.774345 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:34.175353 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:34.175264 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:34.278122 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:34.278085 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:34.427313 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:34.427210 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:35.794876 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:35.794843 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:41.524779 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:41.524742 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:41.632711 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:41.632673 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:41.726285 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:41.726248 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:41.832157 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:41.832068 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:05:42.299620 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:05:42.299589 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:08.200841 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:08.200754 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:14.073099 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:14.073064 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:14.184253 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:14.184217 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:14.274049 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:14.274017 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:30.188100 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:30.188064 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:36.019823 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:36.019788 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:36.129196 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:36.129144 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:36.221311 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:36.221280 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:46.760972 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:46.760938 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:47.638723 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:47.638692 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:53.474003 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:53.473965 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:53.532786 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:53.532736 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:53.640609 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:53.640498 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:53.733378 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:53.733284 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:53.834963 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:53.834929 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:54.594055 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:54.594014 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:54.687865 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:54.687825 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:54.846350 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:54.846259 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:06:56.531298 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:06:56.531267 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:02.314486 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:02.314452 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:02.411622 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:02.411590 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:03.040983 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:03.040950 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:08.977559 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:08.977489 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:09.090898 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.090862 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:09.224431 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.224403 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:09.601138 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.601104 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl"] Jul 16 05:07:09.601565 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.601553 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6380b74-143a-4474-bf20-59c97fb3b639" containerName="istio-proxy" Jul 16 05:07:09.601620 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.601568 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6380b74-143a-4474-bf20-59c97fb3b639" containerName="istio-proxy" Jul 16 05:07:09.601655 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.601643 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6380b74-143a-4474-bf20-59c97fb3b639" containerName="istio-proxy" Jul 16 05:07:09.604776 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.604756 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.615610 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.615581 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ae2b190a-openshift-default-dockercfg-jkzwp\"" Jul 16 05:07:09.615865 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.615841 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-podinfo\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.615936 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.615878 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55313ac3-afb6-4d80-aa9f-743b36a92456-istiod-ca-cert\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.615936 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.615901 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zjsl\" (UniqueName: \"kubernetes.io/projected/55313ac3-afb6-4d80-aa9f-743b36a92456-kube-api-access-8zjsl\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.616038 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.615949 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-workload-certs\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.616038 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.616006 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-token\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.616038 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.616024 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-credential-socket\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.616182 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.616056 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-envoy\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.616182 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.616101 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-data\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.616182 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.616166 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-workload-socket\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.643824 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.643782 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:09.666883 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.666844 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl"] Jul 16 05:07:09.716872 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.716842 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zjsl\" (UniqueName: \"kubernetes.io/projected/55313ac3-afb6-4d80-aa9f-743b36a92456-kube-api-access-8zjsl\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.717076 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.716881 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-workload-certs\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.717076 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.716926 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-token\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.717076 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.716945 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-credential-socket\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.717076 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.716968 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-envoy\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.717076 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.716987 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-data\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.717076 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.717038 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-workload-socket\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.717446 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.717089 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-podinfo\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.717446 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.717119 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55313ac3-afb6-4d80-aa9f-743b36a92456-istiod-ca-cert\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.717572 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.717478 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-workload-certs\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.717646 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.717565 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-data\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.717646 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.717622 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-workload-socket\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.717783 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.717763 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-credential-socket\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.717868 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.717853 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55313ac3-afb6-4d80-aa9f-743b36a92456-istiod-ca-cert\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.719381 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.719348 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-envoy\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.719701 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.719683 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-podinfo\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.734006 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.733961 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zjsl\" (UniqueName: \"kubernetes.io/projected/55313ac3-afb6-4d80-aa9f-743b36a92456-kube-api-access-8zjsl\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.734709 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.734682 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-token\") pod \"e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:09.812331 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.812288 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:09.923137 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:09.923038 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:10.069685 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:10.069656 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl"] Jul 16 05:07:10.070957 ip-10-0-132-245 kubenswrapper[2581]: W0716 05:07:10.070934 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55313ac3_afb6_4d80_aa9f_743b36a92456.slice/crio-8f502c3568c0443ca97b0fd477acfcf1583a328c7e696a73ad8c0c1e544c0517 WatchSource:0}: Error finding container 8f502c3568c0443ca97b0fd477acfcf1583a328c7e696a73ad8c0c1e544c0517: Status 404 returned error can't find the container with id 8f502c3568c0443ca97b0fd477acfcf1583a328c7e696a73ad8c0c1e544c0517 Jul 16 05:07:10.073112 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:10.073082 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:07:10.073208 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:10.073147 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:07:10.073208 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:10.073175 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:07:10.137056 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:10.137023 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" event={"ID":"55313ac3-afb6-4d80-aa9f-743b36a92456","Type":"ContainerStarted","Data":"8f502c3568c0443ca97b0fd477acfcf1583a328c7e696a73ad8c0c1e544c0517"} Jul 16 05:07:11.144688 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:11.144648 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" event={"ID":"55313ac3-afb6-4d80-aa9f-743b36a92456","Type":"ContainerStarted","Data":"8703fbfcfb3ce6697a88f80c9907ccc69af943d32e0a2266a972199398cdfda7"} Jul 16 05:07:11.176808 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:11.176728 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" podStartSLOduration=2.176708497 podStartE2EDuration="2.176708497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:07:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:07:11.175542732 +0000 UTC m=+1683.901011261" watchObservedRunningTime="2026-07-16 05:07:11.176708497 +0000 UTC m=+1683.902177018" Jul 16 05:07:11.924026 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:11.923987 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:11.929414 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:11.929386 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:12.150685 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:12.150644 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:12.151700 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:12.151679 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:12.253090 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:12.253047 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:12.403728 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:12.403696 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:12.631270 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:12.631181 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:13.160166 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:13.160136 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:13.312430 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:13.312398 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:13.474694 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:13.474653 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:17.790280 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:17.790236 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:18.539710 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:18.539671 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:18.626016 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:18.625977 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:19.025264 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:19.025224 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:24.842645 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:24.842608 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:24.914293 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:24.914258 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:25.576819 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:25.576734 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:26.223198 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:26.223168 2581 generic.go:356] "Generic (PLEG): container finished" podID="55313ac3-afb6-4d80-aa9f-743b36a92456" containerID="8703fbfcfb3ce6697a88f80c9907ccc69af943d32e0a2266a972199398cdfda7" exitCode=0 Jul 16 05:07:26.223771 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:26.223253 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" event={"ID":"55313ac3-afb6-4d80-aa9f-743b36a92456","Type":"ContainerDied","Data":"8703fbfcfb3ce6697a88f80c9907ccc69af943d32e0a2266a972199398cdfda7"} Jul 16 05:07:26.223771 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:26.223758 2581 scope.go:117] "RemoveContainer" containerID="8703fbfcfb3ce6697a88f80c9907ccc69af943d32e0a2266a972199398cdfda7" Jul 16 05:07:26.224800 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:26.224772 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:07:26.224877 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:26.224845 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:07:26.224918 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:26.224881 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:07:27.151205 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:27.151162 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:27.229915 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:27.229872 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" event={"ID":"55313ac3-afb6-4d80-aa9f-743b36a92456","Type":"ContainerStarted","Data":"bbe29658e9c9ac70807aa16b99b68c15ea8333ec0bc91d13abcaea6af61fda77"} Jul 16 05:07:27.923494 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:27.923457 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:27.924613 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:27.924588 2581 patch_prober.go:28] interesting pod/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 16 05:07:27.924709 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:27.924650 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" podUID="55313ac3-afb6-4d80-aa9f-743b36a92456" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 16 05:07:28.923975 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:28.923933 2581 patch_prober.go:28] interesting pod/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 16 05:07:28.924469 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:28.924013 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" podUID="55313ac3-afb6-4d80-aa9f-743b36a92456" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 16 05:07:29.923400 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:29.923364 2581 patch_prober.go:28] interesting pod/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 16 05:07:29.923612 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:29.923418 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" podUID="55313ac3-afb6-4d80-aa9f-743b36a92456" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 16 05:07:30.693270 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:30.693234 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:30.923863 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:30.923829 2581 patch_prober.go:28] interesting pod/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 16 05:07:30.924044 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:30.923898 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" podUID="55313ac3-afb6-4d80-aa9f-743b36a92456" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 16 05:07:31.881860 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:31.881829 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:31.927936 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:31.927905 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:31.928333 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:31.928315 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:31.928925 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:31.928911 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:37.494278 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:37.494233 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:37.598197 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:37.598163 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:37.781658 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:37.781557 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl"] Jul 16 05:07:37.781836 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:37.781802 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" podUID="55313ac3-afb6-4d80-aa9f-743b36a92456" containerName="istio-proxy" containerID="cri-o://bbe29658e9c9ac70807aa16b99b68c15ea8333ec0bc91d13abcaea6af61fda77" gracePeriod=30 Jul 16 05:07:37.816144 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:37.816102 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:38.051886 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:38.051786 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:38.227270 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:38.227221 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:38.435954 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:38.435871 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:38.535783 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:38.535742 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:41.900368 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:41.900335 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:42.061947 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:42.061914 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:43.048444 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.048382 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:43.145973 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.145940 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-workload-certs\") pod \"55313ac3-afb6-4d80-aa9f-743b36a92456\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " Jul 16 05:07:43.145973 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.145975 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-data\") pod \"55313ac3-afb6-4d80-aa9f-743b36a92456\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " Jul 16 05:07:43.146247 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.146007 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-podinfo\") pod \"55313ac3-afb6-4d80-aa9f-743b36a92456\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " Jul 16 05:07:43.146247 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.146036 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-token\") pod \"55313ac3-afb6-4d80-aa9f-743b36a92456\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " Jul 16 05:07:43.146247 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.146051 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-credential-socket\") pod \"55313ac3-afb6-4d80-aa9f-743b36a92456\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " Jul 16 05:07:43.146247 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.146066 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-workload-socket\") pod \"55313ac3-afb6-4d80-aa9f-743b36a92456\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " Jul 16 05:07:43.146247 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.146125 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55313ac3-afb6-4d80-aa9f-743b36a92456-istiod-ca-cert\") pod \"55313ac3-afb6-4d80-aa9f-743b36a92456\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " Jul 16 05:07:43.146247 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.146143 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zjsl\" (UniqueName: \"kubernetes.io/projected/55313ac3-afb6-4d80-aa9f-743b36a92456-kube-api-access-8zjsl\") pod \"55313ac3-afb6-4d80-aa9f-743b36a92456\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " Jul 16 05:07:43.146247 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.146187 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-envoy\") pod \"55313ac3-afb6-4d80-aa9f-743b36a92456\" (UID: \"55313ac3-afb6-4d80-aa9f-743b36a92456\") " Jul 16 05:07:43.146629 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.146329 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "55313ac3-afb6-4d80-aa9f-743b36a92456" (UID: "55313ac3-afb6-4d80-aa9f-743b36a92456"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:07:43.146629 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.146473 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "55313ac3-afb6-4d80-aa9f-743b36a92456" (UID: "55313ac3-afb6-4d80-aa9f-743b36a92456"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:07:43.146729 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.146632 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-credential-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:07:43.146729 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.146655 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-workload-certs\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:07:43.146943 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.146912 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55313ac3-afb6-4d80-aa9f-743b36a92456-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "55313ac3-afb6-4d80-aa9f-743b36a92456" (UID: "55313ac3-afb6-4d80-aa9f-743b36a92456"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:07:43.147040 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.146915 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "55313ac3-afb6-4d80-aa9f-743b36a92456" (UID: "55313ac3-afb6-4d80-aa9f-743b36a92456"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:07:43.147040 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.147020 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-data" (OuterVolumeSpecName: "istio-data") pod "55313ac3-afb6-4d80-aa9f-743b36a92456" (UID: "55313ac3-afb6-4d80-aa9f-743b36a92456"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:07:43.148446 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.148422 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "55313ac3-afb6-4d80-aa9f-743b36a92456" (UID: "55313ac3-afb6-4d80-aa9f-743b36a92456"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:07:43.148446 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.148439 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-token" (OuterVolumeSpecName: "istio-token") pod "55313ac3-afb6-4d80-aa9f-743b36a92456" (UID: "55313ac3-afb6-4d80-aa9f-743b36a92456"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:07:43.148610 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.148492 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "55313ac3-afb6-4d80-aa9f-743b36a92456" (UID: "55313ac3-afb6-4d80-aa9f-743b36a92456"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:07:43.148731 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.148710 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55313ac3-afb6-4d80-aa9f-743b36a92456-kube-api-access-8zjsl" (OuterVolumeSpecName: "kube-api-access-8zjsl") pod "55313ac3-afb6-4d80-aa9f-743b36a92456" (UID: "55313ac3-afb6-4d80-aa9f-743b36a92456"). InnerVolumeSpecName "kube-api-access-8zjsl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:07:43.247960 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.247920 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-token\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:07:43.247960 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.247949 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-workload-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:07:43.247960 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.247960 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55313ac3-afb6-4d80-aa9f-743b36a92456-istiod-ca-cert\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:07:43.247960 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.247970 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zjsl\" (UniqueName: \"kubernetes.io/projected/55313ac3-afb6-4d80-aa9f-743b36a92456-kube-api-access-8zjsl\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:07:43.248232 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.247979 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-envoy\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:07:43.248232 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.247988 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-data\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:07:43.248232 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.247995 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55313ac3-afb6-4d80-aa9f-743b36a92456-istio-podinfo\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:07:43.328085 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.327994 2581 generic.go:356] "Generic (PLEG): container finished" podID="55313ac3-afb6-4d80-aa9f-743b36a92456" containerID="bbe29658e9c9ac70807aa16b99b68c15ea8333ec0bc91d13abcaea6af61fda77" exitCode=0 Jul 16 05:07:43.328085 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.328064 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" Jul 16 05:07:43.328270 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.328076 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" event={"ID":"55313ac3-afb6-4d80-aa9f-743b36a92456","Type":"ContainerDied","Data":"bbe29658e9c9ac70807aa16b99b68c15ea8333ec0bc91d13abcaea6af61fda77"} Jul 16 05:07:43.328270 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.328156 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl" event={"ID":"55313ac3-afb6-4d80-aa9f-743b36a92456","Type":"ContainerDied","Data":"8f502c3568c0443ca97b0fd477acfcf1583a328c7e696a73ad8c0c1e544c0517"} Jul 16 05:07:43.328270 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.328178 2581 scope.go:117] "RemoveContainer" containerID="bbe29658e9c9ac70807aa16b99b68c15ea8333ec0bc91d13abcaea6af61fda77" Jul 16 05:07:43.338151 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.338133 2581 scope.go:117] "RemoveContainer" containerID="8703fbfcfb3ce6697a88f80c9907ccc69af943d32e0a2266a972199398cdfda7" Jul 16 05:07:43.346244 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.346215 2581 scope.go:117] "RemoveContainer" containerID="bbe29658e9c9ac70807aa16b99b68c15ea8333ec0bc91d13abcaea6af61fda77" Jul 16 05:07:43.346525 ip-10-0-132-245 kubenswrapper[2581]: E0716 05:07:43.346484 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbe29658e9c9ac70807aa16b99b68c15ea8333ec0bc91d13abcaea6af61fda77\": container with ID starting with bbe29658e9c9ac70807aa16b99b68c15ea8333ec0bc91d13abcaea6af61fda77 not found: ID does not exist" containerID="bbe29658e9c9ac70807aa16b99b68c15ea8333ec0bc91d13abcaea6af61fda77" Jul 16 05:07:43.346582 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.346534 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe29658e9c9ac70807aa16b99b68c15ea8333ec0bc91d13abcaea6af61fda77"} err="failed to get container status \"bbe29658e9c9ac70807aa16b99b68c15ea8333ec0bc91d13abcaea6af61fda77\": rpc error: code = NotFound desc = could not find container \"bbe29658e9c9ac70807aa16b99b68c15ea8333ec0bc91d13abcaea6af61fda77\": container with ID starting with bbe29658e9c9ac70807aa16b99b68c15ea8333ec0bc91d13abcaea6af61fda77 not found: ID does not exist" Jul 16 05:07:43.346582 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.346553 2581 scope.go:117] "RemoveContainer" containerID="8703fbfcfb3ce6697a88f80c9907ccc69af943d32e0a2266a972199398cdfda7" Jul 16 05:07:43.346779 ip-10-0-132-245 kubenswrapper[2581]: E0716 05:07:43.346764 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8703fbfcfb3ce6697a88f80c9907ccc69af943d32e0a2266a972199398cdfda7\": container with ID starting with 8703fbfcfb3ce6697a88f80c9907ccc69af943d32e0a2266a972199398cdfda7 not found: ID does not exist" containerID="8703fbfcfb3ce6697a88f80c9907ccc69af943d32e0a2266a972199398cdfda7" Jul 16 05:07:43.346827 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.346782 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8703fbfcfb3ce6697a88f80c9907ccc69af943d32e0a2266a972199398cdfda7"} err="failed to get container status \"8703fbfcfb3ce6697a88f80c9907ccc69af943d32e0a2266a972199398cdfda7\": rpc error: code = NotFound desc = could not find container \"8703fbfcfb3ce6697a88f80c9907ccc69af943d32e0a2266a972199398cdfda7\": container with ID starting with 8703fbfcfb3ce6697a88f80c9907ccc69af943d32e0a2266a972199398cdfda7 not found: ID does not exist" Jul 16 05:07:43.361724 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.361693 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl"] Jul 16 05:07:43.373594 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.373567 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ae2b190a-openshift-default-6d958d8455-b8lpl"] Jul 16 05:07:43.839296 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:43.839265 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55313ac3-afb6-4d80-aa9f-743b36a92456" path="/var/lib/kubelet/pods/55313ac3-afb6-4d80-aa9f-743b36a92456/volumes" Jul 16 05:07:45.313356 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:45.313318 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:45.416108 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:45.416080 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:45.522894 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:45.522862 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:45.773416 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:45.773382 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:51.523320 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:51.523277 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:51.639434 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:51.639396 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:58.709222 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:58.709178 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:59.194340 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:59.194229 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:59.284260 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:59.284219 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:07:59.383772 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:07:59.383741 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:00.128792 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:00.128725 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:00.240223 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:00.240192 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:00.330219 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:00.330187 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:01.767864 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:01.767827 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:07.509806 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:07.509764 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:07.611557 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:07.611495 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:08.217205 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:08.217168 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:14.140653 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:14.140620 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:14.217409 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:14.217372 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:14.305601 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:14.305566 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:14.403279 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:14.403191 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:24.913594 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:24.913552 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:25.280231 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:25.280198 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:46.431375 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:46.431340 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:52.287154 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:52.287118 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:52.370205 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:52.370171 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:52.673987 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:52.673898 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:52.984664 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:52.984631 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2"] Jul 16 05:08:52.985193 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:52.985175 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55313ac3-afb6-4d80-aa9f-743b36a92456" containerName="istio-proxy" Jul 16 05:08:52.985193 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:52.985195 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="55313ac3-afb6-4d80-aa9f-743b36a92456" containerName="istio-proxy" Jul 16 05:08:52.985357 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:52.985229 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55313ac3-afb6-4d80-aa9f-743b36a92456" containerName="istio-proxy" Jul 16 05:08:52.985357 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:52.985238 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="55313ac3-afb6-4d80-aa9f-743b36a92456" containerName="istio-proxy" Jul 16 05:08:52.985357 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:52.985343 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="55313ac3-afb6-4d80-aa9f-743b36a92456" containerName="istio-proxy" Jul 16 05:08:52.985357 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:52.985356 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="55313ac3-afb6-4d80-aa9f-743b36a92456" containerName="istio-proxy" Jul 16 05:08:52.988529 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:52.988496 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:52.990989 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:52.990969 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-82a682-openshift-default-dockercfg-c96rj\"" Jul 16 05:08:53.006803 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.006779 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2"] Jul 16 05:08:53.016385 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.016341 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:53.109806 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.109771 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-istio-envoy\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.109986 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.109819 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-credential-socket\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.109986 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.109850 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-workload-socket\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.109986 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.109871 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5mn2\" (UniqueName: \"kubernetes.io/projected/ea7fac50-6763-461c-b18e-872e03bc291e-kube-api-access-p5mn2\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.109986 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.109920 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea7fac50-6763-461c-b18e-872e03bc291e-istiod-ca-cert\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.109986 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.109973 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea7fac50-6763-461c-b18e-872e03bc291e-istio-podinfo\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.110152 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.110006 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea7fac50-6763-461c-b18e-872e03bc291e-istio-token\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.110152 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.110026 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-workload-certs\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.110152 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.110082 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-istio-data\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.210934 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.210892 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-istio-envoy\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.211139 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.210957 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-credential-socket\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.211139 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.210992 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-workload-socket\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.211139 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.211014 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5mn2\" (UniqueName: \"kubernetes.io/projected/ea7fac50-6763-461c-b18e-872e03bc291e-kube-api-access-p5mn2\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.211139 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.211085 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea7fac50-6763-461c-b18e-872e03bc291e-istiod-ca-cert\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.211139 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.211122 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea7fac50-6763-461c-b18e-872e03bc291e-istio-podinfo\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.211422 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.211154 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea7fac50-6763-461c-b18e-872e03bc291e-istio-token\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.211422 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.211184 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-workload-certs\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.211422 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.211240 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-istio-data\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.211710 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.211684 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-istio-data\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.211862 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.211840 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-workload-socket\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.211934 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.211748 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-workload-certs\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.212027 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.212001 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-credential-socket\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.212159 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.212097 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea7fac50-6763-461c-b18e-872e03bc291e-istiod-ca-cert\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.213539 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.213517 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-istio-envoy\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.213780 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.213758 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea7fac50-6763-461c-b18e-872e03bc291e-istio-podinfo\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.219237 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.219213 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5mn2\" (UniqueName: \"kubernetes.io/projected/ea7fac50-6763-461c-b18e-872e03bc291e-kube-api-access-p5mn2\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.219575 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.219559 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea7fac50-6763-461c-b18e-872e03bc291e-istio-token\") pod \"e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.302662 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.302571 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:53.459608 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.459578 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2"] Jul 16 05:08:53.462100 ip-10-0-132-245 kubenswrapper[2581]: W0716 05:08:53.462069 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea7fac50_6763_461c_b18e_872e03bc291e.slice/crio-5bdbf9206308d284875716b0f7831b6e24c22e087544901687d56625c07b0ed9 WatchSource:0}: Error finding container 5bdbf9206308d284875716b0f7831b6e24c22e087544901687d56625c07b0ed9: Status 404 returned error can't find the container with id 5bdbf9206308d284875716b0f7831b6e24c22e087544901687d56625c07b0ed9 Jul 16 05:08:53.464557 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.464519 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:08:53.464664 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.464607 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:08:53.464664 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.464647 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:08:53.566556 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.566456 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:53.693942 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.693909 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" event={"ID":"ea7fac50-6763-461c-b18e-872e03bc291e","Type":"ContainerStarted","Data":"35eb0bddcc0c233213629efc8d55b42cf967b14fe77699f4822def5b571e3b45"} Jul 16 05:08:53.694112 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.693959 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" event={"ID":"ea7fac50-6763-461c-b18e-872e03bc291e","Type":"ContainerStarted","Data":"5bdbf9206308d284875716b0f7831b6e24c22e087544901687d56625c07b0ed9"} Jul 16 05:08:53.713353 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:53.713294 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" podStartSLOduration=1.713272052 podStartE2EDuration="1.713272052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:08:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:08:53.712179335 +0000 UTC m=+1786.437647884" watchObservedRunningTime="2026-07-16 05:08:53.713272052 +0000 UTC m=+1786.438740576" Jul 16 05:08:54.303804 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:54.303766 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:54.309095 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:54.309067 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:54.664075 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:54.663983 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:54.701391 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:54.701359 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:54.702330 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:54.702310 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:08:54.786233 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:54.786199 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:54.949379 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:54.949301 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:55.248630 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:55.248600 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:55.422790 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:55.422739 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:08:55.521843 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:08:55.521759 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:09:00.196711 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:00.196633 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:09:00.529411 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:00.529376 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:09:00.807456 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:00.807375 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r"] Jul 16 05:09:00.811093 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:00.811075 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:00.814012 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:00.813985 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-05c504-openshift-default-dockercfg-trlxk\"" Jul 16 05:09:00.824272 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:00.824248 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r"] Jul 16 05:09:00.857529 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:00.856988 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:09:00.989730 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:00.989686 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-credential-socket\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:00.989917 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:00.989750 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-envoy\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:00.989917 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:00.989781 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zgtt\" (UniqueName: \"kubernetes.io/projected/d331dbf0-b643-414e-ba6d-602bf66184f7-kube-api-access-9zgtt\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:00.989917 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:00.989810 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-token\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:00.989917 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:00.989867 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-workload-certs\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:00.989917 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:00.989899 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-workload-socket\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:00.990096 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:00.989966 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-podinfo\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:00.990096 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:00.990001 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-data\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:00.990096 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:00.990023 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d331dbf0-b643-414e-ba6d-602bf66184f7-istiod-ca-cert\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.091019 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.090921 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-data\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.091019 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.090963 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d331dbf0-b643-414e-ba6d-602bf66184f7-istiod-ca-cert\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.091019 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.091018 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-credential-socket\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.091302 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.091041 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-envoy\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.091302 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.091056 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zgtt\" (UniqueName: \"kubernetes.io/projected/d331dbf0-b643-414e-ba6d-602bf66184f7-kube-api-access-9zgtt\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.091302 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.091074 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-token\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.091302 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.091111 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-workload-certs\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.091302 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.091142 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-workload-socket\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.091302 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.091208 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-podinfo\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.091900 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.091407 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-data\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.091900 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.091430 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-credential-socket\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.091900 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.091631 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-workload-certs\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.091900 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.091686 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-workload-socket\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.092314 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.092291 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d331dbf0-b643-414e-ba6d-602bf66184f7-istiod-ca-cert\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.093429 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.093409 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-envoy\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.093700 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.093684 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-podinfo\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.101006 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.100980 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-token\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.101455 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.101418 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zgtt\" (UniqueName: \"kubernetes.io/projected/d331dbf0-b643-414e-ba6d-602bf66184f7-kube-api-access-9zgtt\") pod \"e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.123228 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.123198 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:01.286444 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.286373 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:09:01.330571 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.330540 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r"] Jul 16 05:09:01.332753 ip-10-0-132-245 kubenswrapper[2581]: W0716 05:09:01.332725 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd331dbf0_b643_414e_ba6d_602bf66184f7.slice/crio-aca2efa60391881c2d1bd40be826a893ffb03ebc5328841f82efd68a0a24bd8a WatchSource:0}: Error finding container aca2efa60391881c2d1bd40be826a893ffb03ebc5328841f82efd68a0a24bd8a: Status 404 returned error can't find the container with id aca2efa60391881c2d1bd40be826a893ffb03ebc5328841f82efd68a0a24bd8a Jul 16 05:09:01.335197 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.335167 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:09:01.335283 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.335233 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:09:01.335283 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.335261 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:09:01.739272 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.739229 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" event={"ID":"d331dbf0-b643-414e-ba6d-602bf66184f7","Type":"ContainerStarted","Data":"8695f7b22cc4ad8103f4c1353c739f782ffe6ce5c6ac441aa93ddb37813ee8ce"} Jul 16 05:09:01.739456 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.739289 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" event={"ID":"d331dbf0-b643-414e-ba6d-602bf66184f7","Type":"ContainerStarted","Data":"aca2efa60391881c2d1bd40be826a893ffb03ebc5328841f82efd68a0a24bd8a"} Jul 16 05:09:01.761262 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:01.760983 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" podStartSLOduration=1.760962176 podStartE2EDuration="1.760962176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:09:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:09:01.76028101 +0000 UTC m=+1794.485749531" watchObservedRunningTime="2026-07-16 05:09:01.760962176 +0000 UTC m=+1794.486430699" Jul 16 05:09:02.124064 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:02.123972 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:02.130187 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:02.130157 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:02.746738 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:02.746706 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:02.747926 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:02.747899 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:02.807807 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:02.807769 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:09:02.912167 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:02.911975 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:09:03.007547 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:03.007432 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:09:03.119407 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:03.119372 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:09:03.578330 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:03.578289 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:09:03.677373 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:03.677339 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:09:04.761249 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:04.761216 2581 generic.go:356] "Generic (PLEG): container finished" podID="ea7fac50-6763-461c-b18e-872e03bc291e" containerID="35eb0bddcc0c233213629efc8d55b42cf967b14fe77699f4822def5b571e3b45" exitCode=0 Jul 16 05:09:04.761688 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:04.761292 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" event={"ID":"ea7fac50-6763-461c-b18e-872e03bc291e","Type":"ContainerDied","Data":"35eb0bddcc0c233213629efc8d55b42cf967b14fe77699f4822def5b571e3b45"} Jul 16 05:09:04.761803 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:04.761784 2581 scope.go:117] "RemoveContainer" containerID="35eb0bddcc0c233213629efc8d55b42cf967b14fe77699f4822def5b571e3b45" Jul 16 05:09:04.762904 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:04.762875 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:09:04.763009 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:04.762942 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:09:04.763009 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:04.762972 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:09:05.769798 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:05.769757 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" event={"ID":"ea7fac50-6763-461c-b18e-872e03bc291e","Type":"ContainerStarted","Data":"37390d5055d89bbe7760bcba09a4e293c13ea867b3fd073a2a1ce04b36f1fcc1"} Jul 16 05:09:06.303542 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:06.303477 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:09:06.304776 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:06.304744 2581 patch_prober.go:28] interesting pod/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 16 05:09:06.304929 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:06.304828 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" podUID="ea7fac50-6763-461c-b18e-872e03bc291e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 16 05:09:07.303358 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:07.303320 2581 patch_prober.go:28] interesting pod/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 16 05:09:07.303803 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:07.303379 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" podUID="ea7fac50-6763-461c-b18e-872e03bc291e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 16 05:09:07.883731 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:07.883705 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrbbv_95a3a732-50cf-47f5-9629-0b3d241d017a/ovn-acl-logging/0.log" Jul 16 05:09:07.885330 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:07.885304 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrbbv_95a3a732-50cf-47f5-9629-0b3d241d017a/ovn-acl-logging/0.log" Jul 16 05:09:08.249709 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:08.249648 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:09:08.303658 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:08.303621 2581 patch_prober.go:28] interesting pod/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 16 05:09:08.304203 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:08.303687 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" podUID="ea7fac50-6763-461c-b18e-872e03bc291e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 16 05:09:09.303758 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:09.303718 2581 patch_prober.go:28] interesting pod/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 16 05:09:09.304145 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:09.303781 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" podUID="ea7fac50-6763-461c-b18e-872e03bc291e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 16 05:09:09.702161 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:09.702070 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:09:10.307317 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:10.307282 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:09:10.308378 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:10.308356 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:09:16.350340 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:16.350249 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:09:16.451657 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:16.451615 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:09:16.636388 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:16.636307 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:09:18.400930 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:18.400738 2581 patch_prober.go:28] interesting pod/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": read tcp 10.133.0.2:42398->10.133.0.54:15021: read: connection reset by peer" start-of-body= Jul 16 05:09:18.400930 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:18.400810 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" podUID="d331dbf0-b643-414e-ba6d-602bf66184f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": read tcp 10.133.0.2:42398->10.133.0.54:15021: read: connection reset by peer" Jul 16 05:09:18.842515 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:18.842480 2581 generic.go:356] "Generic (PLEG): container finished" podID="d331dbf0-b643-414e-ba6d-602bf66184f7" containerID="8695f7b22cc4ad8103f4c1353c739f782ffe6ce5c6ac441aa93ddb37813ee8ce" exitCode=0 Jul 16 05:09:18.842697 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:18.842562 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" event={"ID":"d331dbf0-b643-414e-ba6d-602bf66184f7","Type":"ContainerDied","Data":"8695f7b22cc4ad8103f4c1353c739f782ffe6ce5c6ac441aa93ddb37813ee8ce"} Jul 16 05:09:18.843136 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:18.843113 2581 scope.go:117] "RemoveContainer" containerID="8695f7b22cc4ad8103f4c1353c739f782ffe6ce5c6ac441aa93ddb37813ee8ce" Jul 16 05:09:18.844183 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:18.844155 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:09:18.844293 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:18.844228 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:09:18.844293 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:18.844268 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:09:19.854287 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:19.854247 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" event={"ID":"d331dbf0-b643-414e-ba6d-602bf66184f7","Type":"ContainerStarted","Data":"b6fd1442634a21132503cd0cad97ae07afefe6f40cabc7526399be9677fe090c"} Jul 16 05:09:20.123759 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:20.123648 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:20.123922 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:20.123886 2581 patch_prober.go:28] interesting pod/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 16 05:09:20.123978 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:20.123932 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" podUID="d331dbf0-b643-414e-ba6d-602bf66184f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 16 05:09:21.123730 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:21.123689 2581 patch_prober.go:28] interesting pod/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 16 05:09:21.124139 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:21.123750 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" podUID="d331dbf0-b643-414e-ba6d-602bf66184f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 16 05:09:22.124185 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:22.124143 2581 patch_prober.go:28] interesting pod/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 16 05:09:22.124610 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:22.124222 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" podUID="d331dbf0-b643-414e-ba6d-602bf66184f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 16 05:09:23.123944 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:23.123902 2581 patch_prober.go:28] interesting pod/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 16 05:09:23.124150 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:23.123990 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" podUID="d331dbf0-b643-414e-ba6d-602bf66184f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 16 05:09:24.127856 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:24.127825 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:24.128257 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:24.128165 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:24.128888 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:24.128871 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:09:25.516523 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:25.516468 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:09:25.657993 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:25.657948 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:09:41.615139 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:09:41.615101 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:10:01.712934 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:01.712889 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:10:02.710561 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:02.710531 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:10:03.172099 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:03.172014 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:10:08.248841 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:08.248795 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:10:44.653445 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:44.653413 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:10:45.912275 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:45.912244 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:10:51.683065 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:51.683031 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:10:51.781960 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:51.781929 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:10:51.963268 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:51.963235 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2"] Jul 16 05:10:51.963476 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:51.963453 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" podUID="ea7fac50-6763-461c-b18e-872e03bc291e" containerName="istio-proxy" containerID="cri-o://37390d5055d89bbe7760bcba09a4e293c13ea867b3fd073a2a1ce04b36f1fcc1" gracePeriod=30 Jul 16 05:10:51.990916 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:51.990883 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:10:52.229326 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:52.229234 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:10:52.719475 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:52.719439 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:10:55.310851 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:55.310818 2581 patch_prober.go:28] interesting pod/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 05:10:55.311311 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:55.310880 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" podUID="ea7fac50-6763-461c-b18e-872e03bc291e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 05:10:57.229065 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.228994 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:10:57.320793 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.320755 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-workload-certs\") pod \"ea7fac50-6763-461c-b18e-872e03bc291e\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " Jul 16 05:10:57.320966 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.320870 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea7fac50-6763-461c-b18e-872e03bc291e-istio-podinfo\") pod \"ea7fac50-6763-461c-b18e-872e03bc291e\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " Jul 16 05:10:57.320966 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.320927 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-credential-socket\") pod \"ea7fac50-6763-461c-b18e-872e03bc291e\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " Jul 16 05:10:57.320966 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.320960 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-istio-data\") pod \"ea7fac50-6763-461c-b18e-872e03bc291e\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " Jul 16 05:10:57.321152 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.320988 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-workload-socket\") pod \"ea7fac50-6763-461c-b18e-872e03bc291e\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " Jul 16 05:10:57.321152 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.321014 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-istio-envoy\") pod \"ea7fac50-6763-461c-b18e-872e03bc291e\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " Jul 16 05:10:57.321152 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.321037 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea7fac50-6763-461c-b18e-872e03bc291e-istiod-ca-cert\") pod \"ea7fac50-6763-461c-b18e-872e03bc291e\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " Jul 16 05:10:57.321152 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.321060 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea7fac50-6763-461c-b18e-872e03bc291e-istio-token\") pod \"ea7fac50-6763-461c-b18e-872e03bc291e\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " Jul 16 05:10:57.321152 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.321060 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ea7fac50-6763-461c-b18e-872e03bc291e" (UID: "ea7fac50-6763-461c-b18e-872e03bc291e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:10:57.321152 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.321107 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5mn2\" (UniqueName: \"kubernetes.io/projected/ea7fac50-6763-461c-b18e-872e03bc291e-kube-api-access-p5mn2\") pod \"ea7fac50-6763-461c-b18e-872e03bc291e\" (UID: \"ea7fac50-6763-461c-b18e-872e03bc291e\") " Jul 16 05:10:57.321434 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.321317 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ea7fac50-6763-461c-b18e-872e03bc291e" (UID: "ea7fac50-6763-461c-b18e-872e03bc291e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:10:57.321495 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.321435 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea7fac50-6763-461c-b18e-872e03bc291e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ea7fac50-6763-461c-b18e-872e03bc291e" (UID: "ea7fac50-6763-461c-b18e-872e03bc291e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:10:57.321495 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.321473 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-workload-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:10:57.321495 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.321493 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-workload-certs\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:10:57.321692 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.321641 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ea7fac50-6763-461c-b18e-872e03bc291e" (UID: "ea7fac50-6763-461c-b18e-872e03bc291e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:10:57.321901 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.321873 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-istio-data" (OuterVolumeSpecName: "istio-data") pod "ea7fac50-6763-461c-b18e-872e03bc291e" (UID: "ea7fac50-6763-461c-b18e-872e03bc291e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:10:57.323201 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.323174 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea7fac50-6763-461c-b18e-872e03bc291e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ea7fac50-6763-461c-b18e-872e03bc291e" (UID: "ea7fac50-6763-461c-b18e-872e03bc291e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:10:57.323424 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.323400 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea7fac50-6763-461c-b18e-872e03bc291e-kube-api-access-p5mn2" (OuterVolumeSpecName: "kube-api-access-p5mn2") pod "ea7fac50-6763-461c-b18e-872e03bc291e" (UID: "ea7fac50-6763-461c-b18e-872e03bc291e"). InnerVolumeSpecName "kube-api-access-p5mn2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:10:57.323661 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.323635 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea7fac50-6763-461c-b18e-872e03bc291e-istio-token" (OuterVolumeSpecName: "istio-token") pod "ea7fac50-6763-461c-b18e-872e03bc291e" (UID: "ea7fac50-6763-461c-b18e-872e03bc291e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:10:57.323731 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.323677 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ea7fac50-6763-461c-b18e-872e03bc291e" (UID: "ea7fac50-6763-461c-b18e-872e03bc291e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:10:57.358107 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.358070 2581 generic.go:356] "Generic (PLEG): container finished" podID="ea7fac50-6763-461c-b18e-872e03bc291e" containerID="37390d5055d89bbe7760bcba09a4e293c13ea867b3fd073a2a1ce04b36f1fcc1" exitCode=0 Jul 16 05:10:57.358257 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.358116 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" Jul 16 05:10:57.358257 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.358151 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" event={"ID":"ea7fac50-6763-461c-b18e-872e03bc291e","Type":"ContainerDied","Data":"37390d5055d89bbe7760bcba09a4e293c13ea867b3fd073a2a1ce04b36f1fcc1"} Jul 16 05:10:57.358257 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.358232 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2" event={"ID":"ea7fac50-6763-461c-b18e-872e03bc291e","Type":"ContainerDied","Data":"5bdbf9206308d284875716b0f7831b6e24c22e087544901687d56625c07b0ed9"} Jul 16 05:10:57.358257 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.358255 2581 scope.go:117] "RemoveContainer" containerID="37390d5055d89bbe7760bcba09a4e293c13ea867b3fd073a2a1ce04b36f1fcc1" Jul 16 05:10:57.367483 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.367466 2581 scope.go:117] "RemoveContainer" containerID="35eb0bddcc0c233213629efc8d55b42cf967b14fe77699f4822def5b571e3b45" Jul 16 05:10:57.375950 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.375933 2581 scope.go:117] "RemoveContainer" containerID="37390d5055d89bbe7760bcba09a4e293c13ea867b3fd073a2a1ce04b36f1fcc1" Jul 16 05:10:57.376206 ip-10-0-132-245 kubenswrapper[2581]: E0716 05:10:57.376187 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37390d5055d89bbe7760bcba09a4e293c13ea867b3fd073a2a1ce04b36f1fcc1\": container with ID starting with 37390d5055d89bbe7760bcba09a4e293c13ea867b3fd073a2a1ce04b36f1fcc1 not found: ID does not exist" containerID="37390d5055d89bbe7760bcba09a4e293c13ea867b3fd073a2a1ce04b36f1fcc1" Jul 16 05:10:57.376256 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.376216 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37390d5055d89bbe7760bcba09a4e293c13ea867b3fd073a2a1ce04b36f1fcc1"} err="failed to get container status \"37390d5055d89bbe7760bcba09a4e293c13ea867b3fd073a2a1ce04b36f1fcc1\": rpc error: code = NotFound desc = could not find container \"37390d5055d89bbe7760bcba09a4e293c13ea867b3fd073a2a1ce04b36f1fcc1\": container with ID starting with 37390d5055d89bbe7760bcba09a4e293c13ea867b3fd073a2a1ce04b36f1fcc1 not found: ID does not exist" Jul 16 05:10:57.376256 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.376235 2581 scope.go:117] "RemoveContainer" containerID="35eb0bddcc0c233213629efc8d55b42cf967b14fe77699f4822def5b571e3b45" Jul 16 05:10:57.376463 ip-10-0-132-245 kubenswrapper[2581]: E0716 05:10:57.376448 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35eb0bddcc0c233213629efc8d55b42cf967b14fe77699f4822def5b571e3b45\": container with ID starting with 35eb0bddcc0c233213629efc8d55b42cf967b14fe77699f4822def5b571e3b45 not found: ID does not exist" containerID="35eb0bddcc0c233213629efc8d55b42cf967b14fe77699f4822def5b571e3b45" Jul 16 05:10:57.376515 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.376467 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35eb0bddcc0c233213629efc8d55b42cf967b14fe77699f4822def5b571e3b45"} err="failed to get container status \"35eb0bddcc0c233213629efc8d55b42cf967b14fe77699f4822def5b571e3b45\": rpc error: code = NotFound desc = could not find container \"35eb0bddcc0c233213629efc8d55b42cf967b14fe77699f4822def5b571e3b45\": container with ID starting with 35eb0bddcc0c233213629efc8d55b42cf967b14fe77699f4822def5b571e3b45 not found: ID does not exist" Jul 16 05:10:57.381067 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.381045 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2"] Jul 16 05:10:57.385657 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.385635 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-82a682-openshift-default-55d5f644c-cp5r2"] Jul 16 05:10:57.422682 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.422645 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea7fac50-6763-461c-b18e-872e03bc291e-istio-podinfo\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:10:57.422682 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.422676 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-credential-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:10:57.422884 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.422692 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-istio-data\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:10:57.422884 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.422705 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea7fac50-6763-461c-b18e-872e03bc291e-istio-envoy\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:10:57.422884 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.422716 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea7fac50-6763-461c-b18e-872e03bc291e-istiod-ca-cert\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:10:57.422884 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.422727 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea7fac50-6763-461c-b18e-872e03bc291e-istio-token\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:10:57.422884 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.422737 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5mn2\" (UniqueName: \"kubernetes.io/projected/ea7fac50-6763-461c-b18e-872e03bc291e-kube-api-access-p5mn2\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:10:57.840496 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:57.840458 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea7fac50-6763-461c-b18e-872e03bc291e" path="/var/lib/kubelet/pods/ea7fac50-6763-461c-b18e-872e03bc291e/volumes" Jul 16 05:10:58.727561 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:58.727527 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:10:58.857703 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:10:58.857671 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:04.408330 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:04.408294 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:04.513059 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:04.513023 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:04.625910 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:04.625863 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r"] Jul 16 05:11:04.626109 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:04.626082 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" podUID="d331dbf0-b643-414e-ba6d-602bf66184f7" containerName="istio-proxy" containerID="cri-o://b6fd1442634a21132503cd0cad97ae07afefe6f40cabc7526399be9677fe090c" gracePeriod=30 Jul 16 05:11:04.650523 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:04.650479 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:04.904936 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:04.904871 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:05.273234 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.273182 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:05.664428 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.664343 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g"] Jul 16 05:11:05.664795 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.664759 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea7fac50-6763-461c-b18e-872e03bc291e" containerName="istio-proxy" Jul 16 05:11:05.664795 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.664771 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea7fac50-6763-461c-b18e-872e03bc291e" containerName="istio-proxy" Jul 16 05:11:05.664867 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.664846 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea7fac50-6763-461c-b18e-872e03bc291e" containerName="istio-proxy" Jul 16 05:11:05.664867 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.664857 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea7fac50-6763-461c-b18e-872e03bc291e" containerName="istio-proxy" Jul 16 05:11:05.669608 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.669582 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.672626 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.672605 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-e9b55e-openshift-default-dockercfg-8gjdm\"" Jul 16 05:11:05.681301 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.681277 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g"] Jul 16 05:11:05.690542 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.690490 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:05.801121 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.801074 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3859978f-c3d7-4b90-8175-e9bd65156954-istio-token\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.801121 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.801124 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-workload-socket\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.801390 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.801148 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-workload-certs\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.801390 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.801208 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-istio-data\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.801390 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.801242 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-credential-socket\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.801390 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.801262 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x98gc\" (UniqueName: \"kubernetes.io/projected/3859978f-c3d7-4b90-8175-e9bd65156954-kube-api-access-x98gc\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.801390 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.801284 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3859978f-c3d7-4b90-8175-e9bd65156954-istio-podinfo\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.801390 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.801355 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-istio-envoy\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.801755 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.801406 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3859978f-c3d7-4b90-8175-e9bd65156954-istiod-ca-cert\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.897838 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.897801 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:05.902082 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.902058 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3859978f-c3d7-4b90-8175-e9bd65156954-istiod-ca-cert\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.902192 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.902119 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3859978f-c3d7-4b90-8175-e9bd65156954-istio-token\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.902192 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.902142 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-workload-socket\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.902192 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.902163 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-workload-certs\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.902192 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.902188 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-istio-data\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.902402 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.902239 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-credential-socket\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.902402 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.902263 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x98gc\" (UniqueName: \"kubernetes.io/projected/3859978f-c3d7-4b90-8175-e9bd65156954-kube-api-access-x98gc\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.902402 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.902299 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3859978f-c3d7-4b90-8175-e9bd65156954-istio-podinfo\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.902402 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.902339 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-istio-envoy\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.902911 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.902877 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-istio-data\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.903003 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.902923 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3859978f-c3d7-4b90-8175-e9bd65156954-istiod-ca-cert\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.903003 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.902941 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-workload-socket\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.903003 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.902978 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-workload-certs\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.903173 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.903015 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-credential-socket\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.904978 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.904956 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-istio-envoy\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.905106 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.905091 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3859978f-c3d7-4b90-8175-e9bd65156954-istio-podinfo\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.910617 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.910586 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3859978f-c3d7-4b90-8175-e9bd65156954-istio-token\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.910814 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.910796 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x98gc\" (UniqueName: \"kubernetes.io/projected/3859978f-c3d7-4b90-8175-e9bd65156954-kube-api-access-x98gc\") pod \"e2e-api-a-e9b55e-openshift-default-966679697-7l69g\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:05.980980 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:05.980948 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:06.126990 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:06.126966 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g"] Jul 16 05:11:06.129492 ip-10-0-132-245 kubenswrapper[2581]: W0716 05:11:06.129455 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3859978f_c3d7_4b90_8175_e9bd65156954.slice/crio-66ecac6ed976a70d3e916a543228ff9c95ef8adb39a0286ef36abb939869b2a0 WatchSource:0}: Error finding container 66ecac6ed976a70d3e916a543228ff9c95ef8adb39a0286ef36abb939869b2a0: Status 404 returned error can't find the container with id 66ecac6ed976a70d3e916a543228ff9c95ef8adb39a0286ef36abb939869b2a0 Jul 16 05:11:06.131749 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:06.131709 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:11:06.131838 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:06.131813 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:11:06.131881 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:06.131857 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:11:06.408153 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:06.408063 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" event={"ID":"3859978f-c3d7-4b90-8175-e9bd65156954","Type":"ContainerStarted","Data":"72b3f33de6f24b503915bdba48c85c46f57f68061fd3b6611f311e24fac90ea6"} Jul 16 05:11:06.408153 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:06.408130 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" event={"ID":"3859978f-c3d7-4b90-8175-e9bd65156954","Type":"ContainerStarted","Data":"66ecac6ed976a70d3e916a543228ff9c95ef8adb39a0286ef36abb939869b2a0"} Jul 16 05:11:06.429986 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:06.429916 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" podStartSLOduration=1.429894115 podStartE2EDuration="1.429894115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:11:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:11:06.429191228 +0000 UTC m=+1919.154659751" watchObservedRunningTime="2026-07-16 05:11:06.429894115 +0000 UTC m=+1919.155362637" Jul 16 05:11:06.981793 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:06.981755 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:07.405064 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:07.404981 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:07.526239 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:07.526199 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:07.667089 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:07.666999 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:07.817727 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:07.817686 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:07.986436 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:07.986407 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:08.186000 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:08.185963 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:08.289778 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:08.289744 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:08.418122 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:08.418084 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:08.419050 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:08.419031 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:09.130911 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:09.130875 2581 patch_prober.go:28] interesting pod/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 05:11:09.131292 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:09.130941 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" podUID="d331dbf0-b643-414e-ba6d-602bf66184f7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 05:11:09.879667 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:09.879642 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:11:09.932351 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:09.932262 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zgtt\" (UniqueName: \"kubernetes.io/projected/d331dbf0-b643-414e-ba6d-602bf66184f7-kube-api-access-9zgtt\") pod \"d331dbf0-b643-414e-ba6d-602bf66184f7\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " Jul 16 05:11:09.932496 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:09.932370 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-workload-socket\") pod \"d331dbf0-b643-414e-ba6d-602bf66184f7\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " Jul 16 05:11:09.932732 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:09.932706 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d331dbf0-b643-414e-ba6d-602bf66184f7" (UID: "d331dbf0-b643-414e-ba6d-602bf66184f7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:11:09.934417 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:09.934393 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d331dbf0-b643-414e-ba6d-602bf66184f7-kube-api-access-9zgtt" (OuterVolumeSpecName: "kube-api-access-9zgtt") pod "d331dbf0-b643-414e-ba6d-602bf66184f7" (UID: "d331dbf0-b643-414e-ba6d-602bf66184f7"). InnerVolumeSpecName "kube-api-access-9zgtt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:11:10.033369 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.033328 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-token\") pod \"d331dbf0-b643-414e-ba6d-602bf66184f7\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " Jul 16 05:11:10.033581 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.033399 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-credential-socket\") pod \"d331dbf0-b643-414e-ba6d-602bf66184f7\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " Jul 16 05:11:10.033581 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.033424 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d331dbf0-b643-414e-ba6d-602bf66184f7-istiod-ca-cert\") pod \"d331dbf0-b643-414e-ba6d-602bf66184f7\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " Jul 16 05:11:10.033581 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.033466 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-workload-certs\") pod \"d331dbf0-b643-414e-ba6d-602bf66184f7\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " Jul 16 05:11:10.033581 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.033543 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-data\") pod \"d331dbf0-b643-414e-ba6d-602bf66184f7\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " Jul 16 05:11:10.033871 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.033592 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-podinfo\") pod \"d331dbf0-b643-414e-ba6d-602bf66184f7\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " Jul 16 05:11:10.033871 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.033660 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-envoy\") pod \"d331dbf0-b643-414e-ba6d-602bf66184f7\" (UID: \"d331dbf0-b643-414e-ba6d-602bf66184f7\") " Jul 16 05:11:10.033871 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.033751 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d331dbf0-b643-414e-ba6d-602bf66184f7" (UID: "d331dbf0-b643-414e-ba6d-602bf66184f7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:11:10.033871 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.033792 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d331dbf0-b643-414e-ba6d-602bf66184f7" (UID: "d331dbf0-b643-414e-ba6d-602bf66184f7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:11:10.033871 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.033832 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d331dbf0-b643-414e-ba6d-602bf66184f7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d331dbf0-b643-414e-ba6d-602bf66184f7" (UID: "d331dbf0-b643-414e-ba6d-602bf66184f7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:11:10.034074 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.033992 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zgtt\" (UniqueName: \"kubernetes.io/projected/d331dbf0-b643-414e-ba6d-602bf66184f7-kube-api-access-9zgtt\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:11:10.034074 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.034010 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-workload-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:11:10.034074 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.034025 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-credential-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:11:10.034074 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.034038 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d331dbf0-b643-414e-ba6d-602bf66184f7-istiod-ca-cert\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:11:10.034074 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.034051 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-workload-certs\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:11:10.034339 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.034300 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-data" (OuterVolumeSpecName: "istio-data") pod "d331dbf0-b643-414e-ba6d-602bf66184f7" (UID: "d331dbf0-b643-414e-ba6d-602bf66184f7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:11:10.035705 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.035674 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-token" (OuterVolumeSpecName: "istio-token") pod "d331dbf0-b643-414e-ba6d-602bf66184f7" (UID: "d331dbf0-b643-414e-ba6d-602bf66184f7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:11:10.035705 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.035687 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d331dbf0-b643-414e-ba6d-602bf66184f7" (UID: "d331dbf0-b643-414e-ba6d-602bf66184f7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:11:10.035861 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.035758 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d331dbf0-b643-414e-ba6d-602bf66184f7" (UID: "d331dbf0-b643-414e-ba6d-602bf66184f7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:11:10.134920 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.134884 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-data\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:11:10.134920 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.134915 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-podinfo\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:11:10.134920 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.134925 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-envoy\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:11:10.135370 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.134933 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d331dbf0-b643-414e-ba6d-602bf66184f7-istio-token\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:11:10.429043 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.429009 2581 generic.go:356] "Generic (PLEG): container finished" podID="d331dbf0-b643-414e-ba6d-602bf66184f7" containerID="b6fd1442634a21132503cd0cad97ae07afefe6f40cabc7526399be9677fe090c" exitCode=0 Jul 16 05:11:10.429232 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.429055 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" Jul 16 05:11:10.429232 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.429087 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" event={"ID":"d331dbf0-b643-414e-ba6d-602bf66184f7","Type":"ContainerDied","Data":"b6fd1442634a21132503cd0cad97ae07afefe6f40cabc7526399be9677fe090c"} Jul 16 05:11:10.429232 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.429139 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r" event={"ID":"d331dbf0-b643-414e-ba6d-602bf66184f7","Type":"ContainerDied","Data":"aca2efa60391881c2d1bd40be826a893ffb03ebc5328841f82efd68a0a24bd8a"} Jul 16 05:11:10.429232 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.429162 2581 scope.go:117] "RemoveContainer" containerID="b6fd1442634a21132503cd0cad97ae07afefe6f40cabc7526399be9677fe090c" Jul 16 05:11:10.438851 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.438834 2581 scope.go:117] "RemoveContainer" containerID="8695f7b22cc4ad8103f4c1353c739f782ffe6ce5c6ac441aa93ddb37813ee8ce" Jul 16 05:11:10.447081 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.447058 2581 scope.go:117] "RemoveContainer" containerID="b6fd1442634a21132503cd0cad97ae07afefe6f40cabc7526399be9677fe090c" Jul 16 05:11:10.447316 ip-10-0-132-245 kubenswrapper[2581]: E0716 05:11:10.447297 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6fd1442634a21132503cd0cad97ae07afefe6f40cabc7526399be9677fe090c\": container with ID starting with b6fd1442634a21132503cd0cad97ae07afefe6f40cabc7526399be9677fe090c not found: ID does not exist" containerID="b6fd1442634a21132503cd0cad97ae07afefe6f40cabc7526399be9677fe090c" Jul 16 05:11:10.447370 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.447324 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6fd1442634a21132503cd0cad97ae07afefe6f40cabc7526399be9677fe090c"} err="failed to get container status \"b6fd1442634a21132503cd0cad97ae07afefe6f40cabc7526399be9677fe090c\": rpc error: code = NotFound desc = could not find container \"b6fd1442634a21132503cd0cad97ae07afefe6f40cabc7526399be9677fe090c\": container with ID starting with b6fd1442634a21132503cd0cad97ae07afefe6f40cabc7526399be9677fe090c not found: ID does not exist" Jul 16 05:11:10.447370 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.447344 2581 scope.go:117] "RemoveContainer" containerID="8695f7b22cc4ad8103f4c1353c739f782ffe6ce5c6ac441aa93ddb37813ee8ce" Jul 16 05:11:10.447733 ip-10-0-132-245 kubenswrapper[2581]: E0716 05:11:10.447677 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8695f7b22cc4ad8103f4c1353c739f782ffe6ce5c6ac441aa93ddb37813ee8ce\": container with ID starting with 8695f7b22cc4ad8103f4c1353c739f782ffe6ce5c6ac441aa93ddb37813ee8ce not found: ID does not exist" containerID="8695f7b22cc4ad8103f4c1353c739f782ffe6ce5c6ac441aa93ddb37813ee8ce" Jul 16 05:11:10.447733 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.447706 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8695f7b22cc4ad8103f4c1353c739f782ffe6ce5c6ac441aa93ddb37813ee8ce"} err="failed to get container status \"8695f7b22cc4ad8103f4c1353c739f782ffe6ce5c6ac441aa93ddb37813ee8ce\": rpc error: code = NotFound desc = could not find container \"8695f7b22cc4ad8103f4c1353c739f782ffe6ce5c6ac441aa93ddb37813ee8ce\": container with ID starting with 8695f7b22cc4ad8103f4c1353c739f782ffe6ce5c6ac441aa93ddb37813ee8ce not found: ID does not exist" Jul 16 05:11:10.450883 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.450863 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r"] Jul 16 05:11:10.454614 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:10.454594 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-05c504-openshift-default-94dcc5b65-d7t4r"] Jul 16 05:11:11.839012 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:11.838978 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d331dbf0-b643-414e-ba6d-602bf66184f7" path="/var/lib/kubelet/pods/d331dbf0-b643-414e-ba6d-602bf66184f7/volumes" Jul 16 05:11:12.959613 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:12.959583 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:13.435937 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.435817 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:13.754096 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.754055 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw"] Jul 16 05:11:13.754674 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.754655 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d331dbf0-b643-414e-ba6d-602bf66184f7" containerName="istio-proxy" Jul 16 05:11:13.754674 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.754675 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d331dbf0-b643-414e-ba6d-602bf66184f7" containerName="istio-proxy" Jul 16 05:11:13.754808 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.754694 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea7fac50-6763-461c-b18e-872e03bc291e" containerName="istio-proxy" Jul 16 05:11:13.754808 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.754700 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea7fac50-6763-461c-b18e-872e03bc291e" containerName="istio-proxy" Jul 16 05:11:13.754808 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.754790 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d331dbf0-b643-414e-ba6d-602bf66184f7" containerName="istio-proxy" Jul 16 05:11:13.754808 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.754799 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d331dbf0-b643-414e-ba6d-602bf66184f7" containerName="istio-proxy" Jul 16 05:11:13.758326 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.758305 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.760701 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.760683 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-bd340d-openshift-default-dockercfg-9q778\"" Jul 16 05:11:13.763909 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.763888 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-workload-socket\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.764021 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.763919 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-istio-envoy\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.764021 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.763943 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-istio-data\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.764146 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.764041 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18406d68-ab8b-4e94-b673-960a6f212a85-istio-podinfo\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.764146 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.764077 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-credential-socket\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.764146 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.764122 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18406d68-ab8b-4e94-b673-960a6f212a85-istio-token\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.764315 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.764150 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7pjg\" (UniqueName: \"kubernetes.io/projected/18406d68-ab8b-4e94-b673-960a6f212a85-kube-api-access-b7pjg\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.764315 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.764192 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18406d68-ab8b-4e94-b673-960a6f212a85-istiod-ca-cert\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.764315 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.764226 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-workload-certs\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.770577 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.770556 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw"] Jul 16 05:11:13.782130 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.782096 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:13.865326 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.865291 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-istio-envoy\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.865565 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.865334 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-istio-data\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.865565 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.865383 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18406d68-ab8b-4e94-b673-960a6f212a85-istio-podinfo\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.865565 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.865416 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-credential-socket\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.865565 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.865477 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18406d68-ab8b-4e94-b673-960a6f212a85-istio-token\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.865793 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.865564 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7pjg\" (UniqueName: \"kubernetes.io/projected/18406d68-ab8b-4e94-b673-960a6f212a85-kube-api-access-b7pjg\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.865793 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.865615 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18406d68-ab8b-4e94-b673-960a6f212a85-istiod-ca-cert\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.865793 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.865649 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-workload-certs\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.865793 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.865713 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-workload-socket\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.866013 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.865836 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-credential-socket\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.866077 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.866033 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-workload-socket\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.866134 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.866071 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-istio-data\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.866682 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.866623 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-workload-certs\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.866833 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.866758 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18406d68-ab8b-4e94-b673-960a6f212a85-istiod-ca-cert\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.868432 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.868116 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-istio-envoy\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.871277 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.871247 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18406d68-ab8b-4e94-b673-960a6f212a85-istio-podinfo\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.878711 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.878677 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18406d68-ab8b-4e94-b673-960a6f212a85-istio-token\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.879056 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.879040 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7pjg\" (UniqueName: \"kubernetes.io/projected/18406d68-ab8b-4e94-b673-960a6f212a85-kube-api-access-b7pjg\") pod \"e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:13.903417 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:13.903385 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:14.069822 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:14.069727 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:14.220459 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:14.220434 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw"] Jul 16 05:11:14.222941 ip-10-0-132-245 kubenswrapper[2581]: W0716 05:11:14.222918 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18406d68_ab8b_4e94_b673_960a6f212a85.slice/crio-ac8732ff663c0b0046aa2f6f165af41c66beb1a8f14666e7e26fcbdb647fecf4 WatchSource:0}: Error finding container ac8732ff663c0b0046aa2f6f165af41c66beb1a8f14666e7e26fcbdb647fecf4: Status 404 returned error can't find the container with id ac8732ff663c0b0046aa2f6f165af41c66beb1a8f14666e7e26fcbdb647fecf4 Jul 16 05:11:14.225638 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:14.225604 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:11:14.225728 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:14.225709 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:11:14.225796 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:14.225775 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:11:14.458394 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:14.458364 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" event={"ID":"18406d68-ab8b-4e94-b673-960a6f212a85","Type":"ContainerStarted","Data":"9a62ce221ffdf8d5a26075b7207f92447ef51bf540cb6a472412a9c0822721b6"} Jul 16 05:11:14.458620 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:14.458430 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" event={"ID":"18406d68-ab8b-4e94-b673-960a6f212a85","Type":"ContainerStarted","Data":"ac8732ff663c0b0046aa2f6f165af41c66beb1a8f14666e7e26fcbdb647fecf4"} Jul 16 05:11:14.480861 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:14.480809 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" podStartSLOduration=1.480793153 podStartE2EDuration="1.480793153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:11:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:11:14.479176423 +0000 UTC m=+1927.204644946" watchObservedRunningTime="2026-07-16 05:11:14.480793153 +0000 UTC m=+1927.206261692" Jul 16 05:11:15.072218 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:15.070615 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:15.072929 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:15.072285 2581 patch_prober.go:28] interesting pod/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 16 05:11:15.072929 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:15.072339 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" podUID="18406d68-ab8b-4e94-b673-960a6f212a85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 16 05:11:15.453152 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:15.453049 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:15.524070 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:15.524029 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:15.630356 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:15.630320 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:15.792528 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:15.792472 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:16.074432 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:16.074348 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:16.144346 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:16.144225 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:16.244273 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:16.244236 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:16.474667 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:16.474637 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:16.475610 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:16.475586 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:20.965824 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:20.965751 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:21.026640 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:21.026599 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:22.510396 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:22.510366 2581 generic.go:356] "Generic (PLEG): container finished" podID="3859978f-c3d7-4b90-8175-e9bd65156954" containerID="72b3f33de6f24b503915bdba48c85c46f57f68061fd3b6611f311e24fac90ea6" exitCode=0 Jul 16 05:11:22.510805 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:22.510441 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" event={"ID":"3859978f-c3d7-4b90-8175-e9bd65156954","Type":"ContainerDied","Data":"72b3f33de6f24b503915bdba48c85c46f57f68061fd3b6611f311e24fac90ea6"} Jul 16 05:11:22.510888 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:22.510869 2581 scope.go:117] "RemoveContainer" containerID="72b3f33de6f24b503915bdba48c85c46f57f68061fd3b6611f311e24fac90ea6" Jul 16 05:11:22.511938 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:22.511909 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:11:22.512033 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:22.511977 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:11:22.512033 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:22.512007 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:11:23.418298 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:23.418257 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:23.517090 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:23.517056 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" event={"ID":"3859978f-c3d7-4b90-8175-e9bd65156954","Type":"ContainerStarted","Data":"73c406fc9d3980e1d76f3683dd8925341157f83bd454677048d7922d7cf1b103"} Jul 16 05:11:23.981395 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:23.981354 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:23.982823 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:23.982788 2581 patch_prober.go:28] interesting pod/e2e-api-a-e9b55e-openshift-default-966679697-7l69g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 16 05:11:23.982983 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:23.982854 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" podUID="3859978f-c3d7-4b90-8175-e9bd65156954" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 16 05:11:24.982192 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:24.982153 2581 patch_prober.go:28] interesting pod/e2e-api-a-e9b55e-openshift-default-966679697-7l69g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 16 05:11:24.982598 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:24.982209 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" podUID="3859978f-c3d7-4b90-8175-e9bd65156954" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 16 05:11:25.981582 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:25.981539 2581 patch_prober.go:28] interesting pod/e2e-api-a-e9b55e-openshift-default-966679697-7l69g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 16 05:11:25.981784 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:25.981604 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" podUID="3859978f-c3d7-4b90-8175-e9bd65156954" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 16 05:11:26.982231 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:26.981858 2581 patch_prober.go:28] interesting pod/e2e-api-a-e9b55e-openshift-default-966679697-7l69g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 16 05:11:26.982231 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:26.981914 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" podUID="3859978f-c3d7-4b90-8175-e9bd65156954" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 16 05:11:27.985361 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:27.985320 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:27.985782 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:27.985707 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:27.987340 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:27.986494 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:28.564154 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:28.564117 2581 generic.go:356] "Generic (PLEG): container finished" podID="18406d68-ab8b-4e94-b673-960a6f212a85" containerID="9a62ce221ffdf8d5a26075b7207f92447ef51bf540cb6a472412a9c0822721b6" exitCode=0 Jul 16 05:11:28.564366 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:28.564195 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" event={"ID":"18406d68-ab8b-4e94-b673-960a6f212a85","Type":"ContainerDied","Data":"9a62ce221ffdf8d5a26075b7207f92447ef51bf540cb6a472412a9c0822721b6"} Jul 16 05:11:28.564654 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:28.564638 2581 scope.go:117] "RemoveContainer" containerID="9a62ce221ffdf8d5a26075b7207f92447ef51bf540cb6a472412a9c0822721b6" Jul 16 05:11:28.565660 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:28.565628 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:11:28.565757 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:28.565716 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:11:28.565825 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:28.565768 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:11:28.916833 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:28.916743 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:29.016694 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:29.016629 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:29.116726 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:29.116685 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:29.575876 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:29.575839 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" event={"ID":"18406d68-ab8b-4e94-b673-960a6f212a85","Type":"ContainerStarted","Data":"98869c2f160d39c96eabc5d62ef06281e6e339946a0535d683a5a49dffb53270"} Jul 16 05:11:30.070940 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:30.070897 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:30.072121 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:30.072093 2581 patch_prober.go:28] interesting pod/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 16 05:11:30.072239 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:30.072144 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" podUID="18406d68-ab8b-4e94-b673-960a6f212a85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 16 05:11:30.811715 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:30.811677 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:31.071082 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:31.070992 2581 patch_prober.go:28] interesting pod/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 16 05:11:31.071082 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:31.071050 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" podUID="18406d68-ab8b-4e94-b673-960a6f212a85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 16 05:11:31.475188 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:31.475147 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:32.070874 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:32.070833 2581 patch_prober.go:28] interesting pod/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 16 05:11:32.071064 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:32.070895 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" podUID="18406d68-ab8b-4e94-b673-960a6f212a85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 16 05:11:33.071040 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:33.070997 2581 patch_prober.go:28] interesting pod/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 16 05:11:33.071451 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:33.071054 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" podUID="18406d68-ab8b-4e94-b673-960a6f212a85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 16 05:11:34.074182 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:34.074145 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:34.075226 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:34.075204 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:11:36.376301 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:36.376251 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:36.824441 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:36.824405 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:37.156863 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:37.156783 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:37.265097 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:37.265063 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:42.737415 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:42.737380 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:42.802402 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:42.802355 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:43.117655 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:43.117562 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g"] Jul 16 05:11:43.117829 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:43.117776 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" podUID="3859978f-c3d7-4b90-8175-e9bd65156954" containerName="istio-proxy" containerID="cri-o://73c406fc9d3980e1d76f3683dd8925341157f83bd454677048d7922d7cf1b103" gracePeriod=30 Jul 16 05:11:43.128079 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:43.128054 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:43.419949 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:43.419847 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:43.909528 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:43.909474 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:48.385587 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.385525 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:48.422838 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.422802 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-istio-data\") pod \"3859978f-c3d7-4b90-8175-e9bd65156954\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " Jul 16 05:11:48.423028 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.422866 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3859978f-c3d7-4b90-8175-e9bd65156954-istio-token\") pod \"3859978f-c3d7-4b90-8175-e9bd65156954\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " Jul 16 05:11:48.423028 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.422924 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3859978f-c3d7-4b90-8175-e9bd65156954-istio-podinfo\") pod \"3859978f-c3d7-4b90-8175-e9bd65156954\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " Jul 16 05:11:48.423028 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.422972 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-credential-socket\") pod \"3859978f-c3d7-4b90-8175-e9bd65156954\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " Jul 16 05:11:48.423028 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.423003 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-workload-certs\") pod \"3859978f-c3d7-4b90-8175-e9bd65156954\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " Jul 16 05:11:48.423247 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.423033 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-workload-socket\") pod \"3859978f-c3d7-4b90-8175-e9bd65156954\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " Jul 16 05:11:48.423247 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.423098 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3859978f-c3d7-4b90-8175-e9bd65156954-istiod-ca-cert\") pod \"3859978f-c3d7-4b90-8175-e9bd65156954\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " Jul 16 05:11:48.423247 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.423138 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x98gc\" (UniqueName: \"kubernetes.io/projected/3859978f-c3d7-4b90-8175-e9bd65156954-kube-api-access-x98gc\") pod \"3859978f-c3d7-4b90-8175-e9bd65156954\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " Jul 16 05:11:48.423247 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.423169 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-istio-envoy\") pod \"3859978f-c3d7-4b90-8175-e9bd65156954\" (UID: \"3859978f-c3d7-4b90-8175-e9bd65156954\") " Jul 16 05:11:48.423443 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.423387 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3859978f-c3d7-4b90-8175-e9bd65156954" (UID: "3859978f-c3d7-4b90-8175-e9bd65156954"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:11:48.423443 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.423404 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3859978f-c3d7-4b90-8175-e9bd65156954" (UID: "3859978f-c3d7-4b90-8175-e9bd65156954"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:11:48.423583 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.423552 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-workload-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:11:48.423583 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.423573 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-workload-certs\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:11:48.423751 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.423663 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3859978f-c3d7-4b90-8175-e9bd65156954" (UID: "3859978f-c3d7-4b90-8175-e9bd65156954"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:11:48.423822 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.423763 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3859978f-c3d7-4b90-8175-e9bd65156954-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3859978f-c3d7-4b90-8175-e9bd65156954" (UID: "3859978f-c3d7-4b90-8175-e9bd65156954"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:11:48.423939 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.423916 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-istio-data" (OuterVolumeSpecName: "istio-data") pod "3859978f-c3d7-4b90-8175-e9bd65156954" (UID: "3859978f-c3d7-4b90-8175-e9bd65156954"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:11:48.425273 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.425246 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3859978f-c3d7-4b90-8175-e9bd65156954-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3859978f-c3d7-4b90-8175-e9bd65156954" (UID: "3859978f-c3d7-4b90-8175-e9bd65156954"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:11:48.425384 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.425366 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3859978f-c3d7-4b90-8175-e9bd65156954-istio-token" (OuterVolumeSpecName: "istio-token") pod "3859978f-c3d7-4b90-8175-e9bd65156954" (UID: "3859978f-c3d7-4b90-8175-e9bd65156954"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:11:48.425827 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.425802 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3859978f-c3d7-4b90-8175-e9bd65156954" (UID: "3859978f-c3d7-4b90-8175-e9bd65156954"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:11:48.425963 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.425946 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3859978f-c3d7-4b90-8175-e9bd65156954-kube-api-access-x98gc" (OuterVolumeSpecName: "kube-api-access-x98gc") pod "3859978f-c3d7-4b90-8175-e9bd65156954" (UID: "3859978f-c3d7-4b90-8175-e9bd65156954"). InnerVolumeSpecName "kube-api-access-x98gc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:11:48.524074 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.524035 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-istio-data\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:11:48.524074 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.524065 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3859978f-c3d7-4b90-8175-e9bd65156954-istio-token\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:11:48.524074 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.524074 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3859978f-c3d7-4b90-8175-e9bd65156954-istio-podinfo\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:11:48.524074 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.524082 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-credential-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:11:48.524349 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.524091 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3859978f-c3d7-4b90-8175-e9bd65156954-istiod-ca-cert\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:11:48.524349 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.524101 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x98gc\" (UniqueName: \"kubernetes.io/projected/3859978f-c3d7-4b90-8175-e9bd65156954-kube-api-access-x98gc\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:11:48.524349 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.524109 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3859978f-c3d7-4b90-8175-e9bd65156954-istio-envoy\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:11:48.675979 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.675886 2581 generic.go:356] "Generic (PLEG): container finished" podID="3859978f-c3d7-4b90-8175-e9bd65156954" containerID="73c406fc9d3980e1d76f3683dd8925341157f83bd454677048d7922d7cf1b103" exitCode=0 Jul 16 05:11:48.675979 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.675931 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" Jul 16 05:11:48.676167 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.675974 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" event={"ID":"3859978f-c3d7-4b90-8175-e9bd65156954","Type":"ContainerDied","Data":"73c406fc9d3980e1d76f3683dd8925341157f83bd454677048d7922d7cf1b103"} Jul 16 05:11:48.676167 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.676035 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g" event={"ID":"3859978f-c3d7-4b90-8175-e9bd65156954","Type":"ContainerDied","Data":"66ecac6ed976a70d3e916a543228ff9c95ef8adb39a0286ef36abb939869b2a0"} Jul 16 05:11:48.676167 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.676066 2581 scope.go:117] "RemoveContainer" containerID="73c406fc9d3980e1d76f3683dd8925341157f83bd454677048d7922d7cf1b103" Jul 16 05:11:48.685629 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.685612 2581 scope.go:117] "RemoveContainer" containerID="72b3f33de6f24b503915bdba48c85c46f57f68061fd3b6611f311e24fac90ea6" Jul 16 05:11:48.694185 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.694168 2581 scope.go:117] "RemoveContainer" containerID="73c406fc9d3980e1d76f3683dd8925341157f83bd454677048d7922d7cf1b103" Jul 16 05:11:48.694459 ip-10-0-132-245 kubenswrapper[2581]: E0716 05:11:48.694439 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73c406fc9d3980e1d76f3683dd8925341157f83bd454677048d7922d7cf1b103\": container with ID starting with 73c406fc9d3980e1d76f3683dd8925341157f83bd454677048d7922d7cf1b103 not found: ID does not exist" containerID="73c406fc9d3980e1d76f3683dd8925341157f83bd454677048d7922d7cf1b103" Jul 16 05:11:48.694550 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.694472 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73c406fc9d3980e1d76f3683dd8925341157f83bd454677048d7922d7cf1b103"} err="failed to get container status \"73c406fc9d3980e1d76f3683dd8925341157f83bd454677048d7922d7cf1b103\": rpc error: code = NotFound desc = could not find container \"73c406fc9d3980e1d76f3683dd8925341157f83bd454677048d7922d7cf1b103\": container with ID starting with 73c406fc9d3980e1d76f3683dd8925341157f83bd454677048d7922d7cf1b103 not found: ID does not exist" Jul 16 05:11:48.694550 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.694497 2581 scope.go:117] "RemoveContainer" containerID="72b3f33de6f24b503915bdba48c85c46f57f68061fd3b6611f311e24fac90ea6" Jul 16 05:11:48.694751 ip-10-0-132-245 kubenswrapper[2581]: E0716 05:11:48.694733 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72b3f33de6f24b503915bdba48c85c46f57f68061fd3b6611f311e24fac90ea6\": container with ID starting with 72b3f33de6f24b503915bdba48c85c46f57f68061fd3b6611f311e24fac90ea6 not found: ID does not exist" containerID="72b3f33de6f24b503915bdba48c85c46f57f68061fd3b6611f311e24fac90ea6" Jul 16 05:11:48.694794 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.694759 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72b3f33de6f24b503915bdba48c85c46f57f68061fd3b6611f311e24fac90ea6"} err="failed to get container status \"72b3f33de6f24b503915bdba48c85c46f57f68061fd3b6611f311e24fac90ea6\": rpc error: code = NotFound desc = could not find container \"72b3f33de6f24b503915bdba48c85c46f57f68061fd3b6611f311e24fac90ea6\": container with ID starting with 72b3f33de6f24b503915bdba48c85c46f57f68061fd3b6611f311e24fac90ea6 not found: ID does not exist" Jul 16 05:11:48.701604 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.701373 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g"] Jul 16 05:11:48.708287 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:48.708263 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-e9b55e-openshift-default-966679697-7l69g"] Jul 16 05:11:49.559705 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:49.559672 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:49.757641 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:49.757597 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:49.839471 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:49.839385 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3859978f-c3d7-4b90-8175-e9bd65156954" path="/var/lib/kubelet/pods/3859978f-c3d7-4b90-8175-e9bd65156954/volumes" Jul 16 05:11:55.260154 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:55.260108 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:55.361273 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:55.361232 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:55.579992 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:55.579896 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw"] Jul 16 05:11:55.580256 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:55.580228 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" podUID="18406d68-ab8b-4e94-b673-960a6f212a85" containerName="istio-proxy" containerID="cri-o://98869c2f160d39c96eabc5d62ef06281e6e339946a0535d683a5a49dffb53270" gracePeriod=30 Jul 16 05:11:55.604611 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:55.604574 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:55.882897 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:55.882769 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:56.595961 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:56.595926 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:57.222496 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.222460 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89"] Jul 16 05:11:57.223103 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.223087 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3859978f-c3d7-4b90-8175-e9bd65156954" containerName="istio-proxy" Jul 16 05:11:57.223160 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.223107 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3859978f-c3d7-4b90-8175-e9bd65156954" containerName="istio-proxy" Jul 16 05:11:57.223160 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.223129 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d331dbf0-b643-414e-ba6d-602bf66184f7" containerName="istio-proxy" Jul 16 05:11:57.223160 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.223138 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d331dbf0-b643-414e-ba6d-602bf66184f7" containerName="istio-proxy" Jul 16 05:11:57.223160 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.223150 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3859978f-c3d7-4b90-8175-e9bd65156954" containerName="istio-proxy" Jul 16 05:11:57.223282 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.223161 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3859978f-c3d7-4b90-8175-e9bd65156954" containerName="istio-proxy" Jul 16 05:11:57.223282 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.223246 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3859978f-c3d7-4b90-8175-e9bd65156954" containerName="istio-proxy" Jul 16 05:11:57.226635 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.226612 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.229366 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.229342 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a01a82-openshift-default-dockercfg-82pj6\"" Jul 16 05:11:57.237735 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.237705 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89"] Jul 16 05:11:57.241319 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.241296 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:57.309801 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.309746 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-envoy\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.309801 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.309804 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/613495fb-b79a-4d1f-b786-e20f824cc7a6-istiod-ca-cert\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.310089 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.309842 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-workload-certs\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.310089 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.309981 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-credential-socket\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.310089 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.310047 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-data\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.310265 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.310137 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-token\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.310265 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.310167 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-workload-socket\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.310265 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.310214 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfl79\" (UniqueName: \"kubernetes.io/projected/613495fb-b79a-4d1f-b786-e20f824cc7a6-kube-api-access-cfl79\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.310265 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.310256 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-podinfo\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.410965 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.410904 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-credential-socket\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.410965 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.410973 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-data\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.411257 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.411018 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-token\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.411257 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.411040 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-workload-socket\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.411257 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.411089 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfl79\" (UniqueName: \"kubernetes.io/projected/613495fb-b79a-4d1f-b786-e20f824cc7a6-kube-api-access-cfl79\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.411257 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.411125 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-podinfo\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.411257 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.411183 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-envoy\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.411257 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.411211 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/613495fb-b79a-4d1f-b786-e20f824cc7a6-istiod-ca-cert\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.411257 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.411248 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-workload-certs\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.411625 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.411383 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-credential-socket\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.411625 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.411432 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-workload-socket\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.411625 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.411461 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-data\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.411762 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.411661 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-workload-certs\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.412021 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.412003 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/613495fb-b79a-4d1f-b786-e20f824cc7a6-istiod-ca-cert\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.414233 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.414211 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-envoy\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.414410 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.414394 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-podinfo\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.420086 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.420062 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-token\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.420336 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.420314 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfl79\" (UniqueName: \"kubernetes.io/projected/613495fb-b79a-4d1f-b786-e20f824cc7a6-kube-api-access-cfl79\") pod \"e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.540767 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.540671 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:57.689995 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.689964 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89"] Jul 16 05:11:57.693093 ip-10-0-132-245 kubenswrapper[2581]: W0716 05:11:57.693068 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod613495fb_b79a_4d1f_b786_e20f824cc7a6.slice/crio-e6bb446d6727454e20091950dc8e25a9ebe69798b5afdb28dd26134a4ac40bc5 WatchSource:0}: Error finding container e6bb446d6727454e20091950dc8e25a9ebe69798b5afdb28dd26134a4ac40bc5: Status 404 returned error can't find the container with id e6bb446d6727454e20091950dc8e25a9ebe69798b5afdb28dd26134a4ac40bc5 Jul 16 05:11:57.695655 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.695624 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:11:57.695729 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.695695 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:11:57.695789 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.695723 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:11:57.721965 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.721930 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" event={"ID":"613495fb-b79a-4d1f-b786-e20f824cc7a6","Type":"ContainerStarted","Data":"e6bb446d6727454e20091950dc8e25a9ebe69798b5afdb28dd26134a4ac40bc5"} Jul 16 05:11:57.843319 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:57.843223 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:58.728463 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:58.728423 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" event={"ID":"613495fb-b79a-4d1f-b786-e20f824cc7a6","Type":"ContainerStarted","Data":"9629c1a4d9cc91856d95646c7970a2a43e72443b84b26285914d40ecc2cd4ab3"} Jul 16 05:11:58.750053 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:58.750000 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" podStartSLOduration=1.749981757 podStartE2EDuration="1.749981757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:11:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:11:58.748438921 +0000 UTC m=+1971.473907443" watchObservedRunningTime="2026-07-16 05:11:58.749981757 +0000 UTC m=+1971.475450278" Jul 16 05:11:59.540982 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:59.540944 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:59.546587 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:59.546556 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:59.714883 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:59.714811 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:59.734657 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:59.734625 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:59.735740 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:59.735719 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:11:59.810834 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:59.810750 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:11:59.930872 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:11:59.930842 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:00.090665 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.090549 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:00.535974 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.535936 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:00.634692 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.634660 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:00.744497 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.744464 2581 generic.go:356] "Generic (PLEG): container finished" podID="18406d68-ab8b-4e94-b673-960a6f212a85" containerID="98869c2f160d39c96eabc5d62ef06281e6e339946a0535d683a5a49dffb53270" exitCode=0 Jul 16 05:12:00.744950 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.744551 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" event={"ID":"18406d68-ab8b-4e94-b673-960a6f212a85","Type":"ContainerDied","Data":"98869c2f160d39c96eabc5d62ef06281e6e339946a0535d683a5a49dffb53270"} Jul 16 05:12:00.744950 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.744646 2581 scope.go:117] "RemoveContainer" containerID="9a62ce221ffdf8d5a26075b7207f92447ef51bf540cb6a472412a9c0822721b6" Jul 16 05:12:00.850382 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.850356 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:12:00.950549 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.950481 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18406d68-ab8b-4e94-b673-960a6f212a85-istio-podinfo\") pod \"18406d68-ab8b-4e94-b673-960a6f212a85\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " Jul 16 05:12:00.950549 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.950549 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-workload-certs\") pod \"18406d68-ab8b-4e94-b673-960a6f212a85\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " Jul 16 05:12:00.950799 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.950569 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18406d68-ab8b-4e94-b673-960a6f212a85-istiod-ca-cert\") pod \"18406d68-ab8b-4e94-b673-960a6f212a85\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " Jul 16 05:12:00.950799 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.950592 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-istio-envoy\") pod \"18406d68-ab8b-4e94-b673-960a6f212a85\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " Jul 16 05:12:00.950799 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.950657 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-workload-socket\") pod \"18406d68-ab8b-4e94-b673-960a6f212a85\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " Jul 16 05:12:00.950799 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.950689 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7pjg\" (UniqueName: \"kubernetes.io/projected/18406d68-ab8b-4e94-b673-960a6f212a85-kube-api-access-b7pjg\") pod \"18406d68-ab8b-4e94-b673-960a6f212a85\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " Jul 16 05:12:00.950799 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.950717 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-istio-data\") pod \"18406d68-ab8b-4e94-b673-960a6f212a85\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " Jul 16 05:12:00.950799 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.950765 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-credential-socket\") pod \"18406d68-ab8b-4e94-b673-960a6f212a85\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " Jul 16 05:12:00.951089 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.950860 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "18406d68-ab8b-4e94-b673-960a6f212a85" (UID: "18406d68-ab8b-4e94-b673-960a6f212a85"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:12:00.951089 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.950933 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "18406d68-ab8b-4e94-b673-960a6f212a85" (UID: "18406d68-ab8b-4e94-b673-960a6f212a85"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:12:00.951089 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.950966 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18406d68-ab8b-4e94-b673-960a6f212a85-istio-token\") pod \"18406d68-ab8b-4e94-b673-960a6f212a85\" (UID: \"18406d68-ab8b-4e94-b673-960a6f212a85\") " Jul 16 05:12:00.951223 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.951086 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18406d68-ab8b-4e94-b673-960a6f212a85-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "18406d68-ab8b-4e94-b673-960a6f212a85" (UID: "18406d68-ab8b-4e94-b673-960a6f212a85"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:12:00.951223 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.951166 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "18406d68-ab8b-4e94-b673-960a6f212a85" (UID: "18406d68-ab8b-4e94-b673-960a6f212a85"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:12:00.951445 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.951422 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-workload-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:12:00.951445 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.951447 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-credential-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:12:00.951660 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.951462 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-workload-certs\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:12:00.951660 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.951475 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18406d68-ab8b-4e94-b673-960a6f212a85-istiod-ca-cert\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:12:00.951660 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.951556 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-istio-data" (OuterVolumeSpecName: "istio-data") pod "18406d68-ab8b-4e94-b673-960a6f212a85" (UID: "18406d68-ab8b-4e94-b673-960a6f212a85"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:12:00.953088 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.953052 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "18406d68-ab8b-4e94-b673-960a6f212a85" (UID: "18406d68-ab8b-4e94-b673-960a6f212a85"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:12:00.953195 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.953166 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18406d68-ab8b-4e94-b673-960a6f212a85-kube-api-access-b7pjg" (OuterVolumeSpecName: "kube-api-access-b7pjg") pod "18406d68-ab8b-4e94-b673-960a6f212a85" (UID: "18406d68-ab8b-4e94-b673-960a6f212a85"). InnerVolumeSpecName "kube-api-access-b7pjg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:12:00.953326 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.953302 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/18406d68-ab8b-4e94-b673-960a6f212a85-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "18406d68-ab8b-4e94-b673-960a6f212a85" (UID: "18406d68-ab8b-4e94-b673-960a6f212a85"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:12:00.953526 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:00.953492 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18406d68-ab8b-4e94-b673-960a6f212a85-istio-token" (OuterVolumeSpecName: "istio-token") pod "18406d68-ab8b-4e94-b673-960a6f212a85" (UID: "18406d68-ab8b-4e94-b673-960a6f212a85"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:12:01.052556 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:01.052493 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18406d68-ab8b-4e94-b673-960a6f212a85-istio-token\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:12:01.052556 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:01.052555 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18406d68-ab8b-4e94-b673-960a6f212a85-istio-podinfo\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:12:01.052556 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:01.052564 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-istio-envoy\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:12:01.052556 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:01.052574 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7pjg\" (UniqueName: \"kubernetes.io/projected/18406d68-ab8b-4e94-b673-960a6f212a85-kube-api-access-b7pjg\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:12:01.052925 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:01.052584 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18406d68-ab8b-4e94-b673-960a6f212a85-istio-data\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:12:01.751591 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:01.751558 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" Jul 16 05:12:01.752073 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:01.751614 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw" event={"ID":"18406d68-ab8b-4e94-b673-960a6f212a85","Type":"ContainerDied","Data":"ac8732ff663c0b0046aa2f6f165af41c66beb1a8f14666e7e26fcbdb647fecf4"} Jul 16 05:12:01.752073 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:01.751672 2581 scope.go:117] "RemoveContainer" containerID="98869c2f160d39c96eabc5d62ef06281e6e339946a0535d683a5a49dffb53270" Jul 16 05:12:01.779876 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:01.779839 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw"] Jul 16 05:12:01.784962 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:01.784934 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-bd340d-openshift-default-55ff75d979-dqkhw"] Jul 16 05:12:01.841942 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:01.841905 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18406d68-ab8b-4e94-b673-960a6f212a85" path="/var/lib/kubelet/pods/18406d68-ab8b-4e94-b673-960a6f212a85/volumes" Jul 16 05:12:05.268089 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:05.268053 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:05.657350 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:05.657243 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:05.974221 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:05.974169 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498"] Jul 16 05:12:05.974874 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:05.974852 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18406d68-ab8b-4e94-b673-960a6f212a85" containerName="istio-proxy" Jul 16 05:12:05.974973 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:05.974878 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="18406d68-ab8b-4e94-b673-960a6f212a85" containerName="istio-proxy" Jul 16 05:12:05.974973 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:05.974899 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18406d68-ab8b-4e94-b673-960a6f212a85" containerName="istio-proxy" Jul 16 05:12:05.974973 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:05.974908 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="18406d68-ab8b-4e94-b673-960a6f212a85" containerName="istio-proxy" Jul 16 05:12:05.975085 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:05.975003 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3859978f-c3d7-4b90-8175-e9bd65156954" containerName="istio-proxy" Jul 16 05:12:05.975085 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:05.975017 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="18406d68-ab8b-4e94-b673-960a6f212a85" containerName="istio-proxy" Jul 16 05:12:05.975085 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:05.975027 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="18406d68-ab8b-4e94-b673-960a6f212a85" containerName="istio-proxy" Jul 16 05:12:05.978835 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:05.978813 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:05.983229 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:05.983199 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-12efc5-openshift-default-dockercfg-24g7q\"" Jul 16 05:12:05.992662 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:05.992635 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498"] Jul 16 05:12:05.998428 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:05.998390 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:06.098126 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.098089 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ab1851e-c352-470b-9278-3136d221f2be-istio-token\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.098333 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.098233 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ab1851e-c352-470b-9278-3136d221f2be-istiod-ca-cert\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.098333 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.098295 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4pbr\" (UniqueName: \"kubernetes.io/projected/7ab1851e-c352-470b-9278-3136d221f2be-kube-api-access-f4pbr\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.098468 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.098375 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-workload-socket\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.098468 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.098403 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-istio-data\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.098468 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.098432 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-credential-socket\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.098633 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.098534 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ab1851e-c352-470b-9278-3136d221f2be-istio-podinfo\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.098633 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.098585 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-istio-envoy\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.098721 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.098647 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-workload-certs\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.199307 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.199268 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ab1851e-c352-470b-9278-3136d221f2be-istiod-ca-cert\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.199307 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.199314 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4pbr\" (UniqueName: \"kubernetes.io/projected/7ab1851e-c352-470b-9278-3136d221f2be-kube-api-access-f4pbr\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.199657 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.199354 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-workload-socket\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.199657 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.199373 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-istio-data\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.199657 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.199391 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-credential-socket\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.199657 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.199423 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ab1851e-c352-470b-9278-3136d221f2be-istio-podinfo\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.199894 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.199465 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-istio-envoy\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.199894 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.199800 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-workload-certs\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.199894 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.199830 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ab1851e-c352-470b-9278-3136d221f2be-istio-token\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.200168 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.200139 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-istio-data\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.200253 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.200154 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ab1851e-c352-470b-9278-3136d221f2be-istiod-ca-cert\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.200253 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.200228 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-credential-socket\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.200330 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.200305 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-workload-certs\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.200365 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.200313 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-workload-socket\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.202133 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.202112 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-istio-envoy\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.202262 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.202231 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ab1851e-c352-470b-9278-3136d221f2be-istio-podinfo\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.208168 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.208146 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ab1851e-c352-470b-9278-3136d221f2be-istio-token\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.208479 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.208456 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4pbr\" (UniqueName: \"kubernetes.io/projected/7ab1851e-c352-470b-9278-3136d221f2be-kube-api-access-f4pbr\") pod \"e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.289770 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.289678 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:06.292767 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.292742 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:06.479876 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.479844 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498"] Jul 16 05:12:06.481886 ip-10-0-132-245 kubenswrapper[2581]: W0716 05:12:06.481857 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ab1851e_c352_470b_9278_3136d221f2be.slice/crio-ab09ddef4ecf15599361dce68c43232fdd3e89fe01411e255fcb6b45cd7344cf WatchSource:0}: Error finding container ab09ddef4ecf15599361dce68c43232fdd3e89fe01411e255fcb6b45cd7344cf: Status 404 returned error can't find the container with id ab09ddef4ecf15599361dce68c43232fdd3e89fe01411e255fcb6b45cd7344cf Jul 16 05:12:06.484039 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.484008 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:12:06.484117 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.484073 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:12:06.484117 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.484107 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:12:06.785662 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.785626 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" event={"ID":"7ab1851e-c352-470b-9278-3136d221f2be","Type":"ContainerStarted","Data":"24943588ff5777288afc5093922ba0d0a318e7b16d2d2847bd3e085900b04f8d"} Jul 16 05:12:06.785873 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.785689 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" event={"ID":"7ab1851e-c352-470b-9278-3136d221f2be","Type":"ContainerStarted","Data":"ab09ddef4ecf15599361dce68c43232fdd3e89fe01411e255fcb6b45cd7344cf"} Jul 16 05:12:06.808053 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:06.807954 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" podStartSLOduration=1.807934166 podStartE2EDuration="1.807934166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:12:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:12:06.805087891 +0000 UTC m=+1979.530556434" watchObservedRunningTime="2026-07-16 05:12:06.807934166 +0000 UTC m=+1979.533402689" Jul 16 05:12:07.290480 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:07.290440 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:07.295702 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:07.295673 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:07.790901 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:07.790867 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:07.791961 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:07.791937 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:07.960088 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:07.960053 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:08.054059 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:08.053916 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:08.141109 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:08.141071 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:08.679877 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:08.679842 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:08.819627 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:08.819593 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:11.813565 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:11.813538 2581 generic.go:356] "Generic (PLEG): container finished" podID="613495fb-b79a-4d1f-b786-e20f824cc7a6" containerID="9629c1a4d9cc91856d95646c7970a2a43e72443b84b26285914d40ecc2cd4ab3" exitCode=0 Jul 16 05:12:11.813880 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:11.813611 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" event={"ID":"613495fb-b79a-4d1f-b786-e20f824cc7a6","Type":"ContainerDied","Data":"9629c1a4d9cc91856d95646c7970a2a43e72443b84b26285914d40ecc2cd4ab3"} Jul 16 05:12:11.814049 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:11.814034 2581 scope.go:117] "RemoveContainer" containerID="9629c1a4d9cc91856d95646c7970a2a43e72443b84b26285914d40ecc2cd4ab3" Jul 16 05:12:11.815105 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:11.815073 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:12:11.815180 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:11.815163 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:12:11.815217 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:11.815205 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:12:12.820367 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:12.820332 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" event={"ID":"613495fb-b79a-4d1f-b786-e20f824cc7a6","Type":"ContainerStarted","Data":"a81296eaf06e846802bcb60d9f8aca356683ca0e39d617d25458ab167fdfc55e"} Jul 16 05:12:13.553600 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:13.553560 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:12:13.561688 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:13.561651 2581 patch_prober.go:28] interesting pod/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 16 05:12:13.561887 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:13.561717 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" podUID="613495fb-b79a-4d1f-b786-e20f824cc7a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 16 05:12:13.562859 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:13.562832 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:14.541658 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:14.541614 2581 patch_prober.go:28] interesting pod/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 16 05:12:14.542173 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:14.541722 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" podUID="613495fb-b79a-4d1f-b786-e20f824cc7a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 16 05:12:14.734967 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:14.734921 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:12:15.541855 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:15.541807 2581 patch_prober.go:28] interesting pod/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 16 05:12:15.542380 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:15.541890 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" podUID="613495fb-b79a-4d1f-b786-e20f824cc7a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 16 05:12:16.542069 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:16.542023 2581 patch_prober.go:28] interesting pod/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 16 05:12:16.542606 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:16.542101 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" podUID="613495fb-b79a-4d1f-b786-e20f824cc7a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 16 05:12:17.559025 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:17.558981 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:12:17.562815 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:17.562787 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:12:21.343278 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:21.343220 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:21.443720 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:21.443664 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:23.736810 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:23.736712 2581 patch_prober.go:28] interesting pod/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": read tcp 10.133.0.2:44538->10.133.0.58:15021: read: connection reset by peer" start-of-body= Jul 16 05:12:23.737231 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:23.736801 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" podUID="7ab1851e-c352-470b-9278-3136d221f2be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": read tcp 10.133.0.2:44538->10.133.0.58:15021: read: connection reset by peer" Jul 16 05:12:23.897429 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:23.897343 2581 generic.go:356] "Generic (PLEG): container finished" podID="7ab1851e-c352-470b-9278-3136d221f2be" containerID="24943588ff5777288afc5093922ba0d0a318e7b16d2d2847bd3e085900b04f8d" exitCode=0 Jul 16 05:12:23.897429 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:23.897400 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" event={"ID":"7ab1851e-c352-470b-9278-3136d221f2be","Type":"ContainerDied","Data":"24943588ff5777288afc5093922ba0d0a318e7b16d2d2847bd3e085900b04f8d"} Jul 16 05:12:23.897805 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:23.897791 2581 scope.go:117] "RemoveContainer" containerID="24943588ff5777288afc5093922ba0d0a318e7b16d2d2847bd3e085900b04f8d" Jul 16 05:12:23.898826 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:23.898799 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:12:23.898886 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:23.898867 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:12:23.898932 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:23.898895 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 05:12:24.905168 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:24.905126 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" event={"ID":"7ab1851e-c352-470b-9278-3136d221f2be","Type":"ContainerStarted","Data":"06223757d328576c51c77b029d89d9af97970147831d684ea9cdee7531fe8038"} Jul 16 05:12:25.290848 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:25.290809 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:25.291119 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:25.291094 2581 patch_prober.go:28] interesting pod/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 16 05:12:25.291172 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:25.291153 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" podUID="7ab1851e-c352-470b-9278-3136d221f2be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 16 05:12:26.036745 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:26.036698 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:26.095995 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:26.095963 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:26.290569 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:26.290458 2581 patch_prober.go:28] interesting pod/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 16 05:12:26.290569 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:26.290559 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" podUID="7ab1851e-c352-470b-9278-3136d221f2be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 16 05:12:27.290808 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:27.290770 2581 patch_prober.go:28] interesting pod/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 16 05:12:27.291206 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:27.290830 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" podUID="7ab1851e-c352-470b-9278-3136d221f2be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 16 05:12:27.663234 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:27.663129 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:28.290044 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:28.289998 2581 patch_prober.go:28] interesting pod/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 16 05:12:28.290248 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:28.290075 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" podUID="7ab1851e-c352-470b-9278-3136d221f2be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 16 05:12:29.294163 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:29.294131 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:29.294625 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:29.294530 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:29.295179 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:29.295162 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:12:29.536346 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:29.536312 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:29.637664 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:29.637572 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:42.836961 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:42.836915 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:42.936814 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:42.936782 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:12:48.825781 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:12:48.825742 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:13:58.931787 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:13:58.931747 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:14:00.949049 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:14:00.949008 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:14:07.914252 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:14:07.914220 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrbbv_95a3a732-50cf-47f5-9629-0b3d241d017a/ovn-acl-logging/0.log" Jul 16 05:14:07.917589 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:14:07.917564 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrbbv_95a3a732-50cf-47f5-9629-0b3d241d017a/ovn-acl-logging/0.log" Jul 16 05:14:40.043323 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:14:40.043291 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:14:41.110358 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:14:41.110321 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:15:00.370681 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:15:00.370599 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:15:22.632457 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:15:22.632425 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:15:49.345852 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:15:49.345809 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:16:06.393682 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:06.393614 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:16:07.328898 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:07.328867 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:16:09.899278 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:09.899244 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:16:11.896061 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:11.896028 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt"] Jul 16 05:16:11.898601 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:11.898578 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:11.902090 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:11.902065 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a01a82\"/\"kube-root-ca.crt\"" Jul 16 05:16:11.902232 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:11.902069 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a01a82\"/\"rate-test-model-a01a82-kserve-self-signed-certs\"" Jul 16 05:16:11.902232 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:11.902079 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a01a82\"/\"openshift-service-ca.crt\"" Jul 16 05:16:11.902385 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:11.902235 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a01a82\"/\"default-dockercfg-xm6kx\"" Jul 16 05:16:11.908451 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:11.908426 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt"] Jul 16 05:16:11.943460 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:11.943423 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-tls-certs\") pod \"rate-test-model-a01a82-kserve-6ffc45948c-zk6qt\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:11.943649 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:11.943467 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-model-cache\") pod \"rate-test-model-a01a82-kserve-6ffc45948c-zk6qt\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:11.943649 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:11.943492 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-tmp-dir\") pod \"rate-test-model-a01a82-kserve-6ffc45948c-zk6qt\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:11.943649 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:11.943613 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-home\") pod \"rate-test-model-a01a82-kserve-6ffc45948c-zk6qt\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:11.943649 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:11.943640 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-dshm\") pod \"rate-test-model-a01a82-kserve-6ffc45948c-zk6qt\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:11.943813 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:11.943793 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptdk5\" (UniqueName: \"kubernetes.io/projected/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-kube-api-access-ptdk5\") pod \"rate-test-model-a01a82-kserve-6ffc45948c-zk6qt\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:11.966319 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:11.966279 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:16:12.045143 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:12.045104 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptdk5\" (UniqueName: \"kubernetes.io/projected/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-kube-api-access-ptdk5\") pod \"rate-test-model-a01a82-kserve-6ffc45948c-zk6qt\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:12.045320 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:12.045150 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-tls-certs\") pod \"rate-test-model-a01a82-kserve-6ffc45948c-zk6qt\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:12.045374 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:12.045336 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-model-cache\") pod \"rate-test-model-a01a82-kserve-6ffc45948c-zk6qt\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:12.045410 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:12.045389 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-tmp-dir\") pod \"rate-test-model-a01a82-kserve-6ffc45948c-zk6qt\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:12.045468 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:12.045443 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-home\") pod \"rate-test-model-a01a82-kserve-6ffc45948c-zk6qt\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:12.045555 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:12.045480 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-dshm\") pod \"rate-test-model-a01a82-kserve-6ffc45948c-zk6qt\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:12.045768 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:12.045748 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-model-cache\") pod \"rate-test-model-a01a82-kserve-6ffc45948c-zk6qt\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:12.045855 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:12.045815 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-tmp-dir\") pod \"rate-test-model-a01a82-kserve-6ffc45948c-zk6qt\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:12.045855 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:12.045844 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-home\") pod \"rate-test-model-a01a82-kserve-6ffc45948c-zk6qt\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:12.047650 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:12.047623 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-dshm\") pod \"rate-test-model-a01a82-kserve-6ffc45948c-zk6qt\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:12.047946 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:12.047930 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-tls-certs\") pod \"rate-test-model-a01a82-kserve-6ffc45948c-zk6qt\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:12.053845 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:12.053817 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptdk5\" (UniqueName: \"kubernetes.io/projected/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-kube-api-access-ptdk5\") pod \"rate-test-model-a01a82-kserve-6ffc45948c-zk6qt\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:12.209754 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:12.209700 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:12.353323 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:12.353293 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt"] Jul 16 05:16:12.356075 ip-10-0-132-245 kubenswrapper[2581]: W0716 05:16:12.356048 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6d573c3_79e8_4b1c_9c14_cfbbaf1f6dc3.slice/crio-2b016bf6fd54f7ddc82350c998d432012e329e180900da940d339d2c66c7f50e WatchSource:0}: Error finding container 2b016bf6fd54f7ddc82350c998d432012e329e180900da940d339d2c66c7f50e: Status 404 returned error can't find the container with id 2b016bf6fd54f7ddc82350c998d432012e329e180900da940d339d2c66c7f50e Jul 16 05:16:13.108429 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:13.108391 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" event={"ID":"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3","Type":"ContainerStarted","Data":"5085ff87aa7a8b2dd6b9250cc474a4cc4226d1c814683e88dbb231688343800c"} Jul 16 05:16:13.108871 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:13.108436 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" event={"ID":"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3","Type":"ContainerStarted","Data":"2b016bf6fd54f7ddc82350c998d432012e329e180900da940d339d2c66c7f50e"} Jul 16 05:16:13.125416 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:13.125353 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" podStartSLOduration=2.125335555 podStartE2EDuration="2.125335555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:16:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:16:13.124497639 +0000 UTC m=+2225.849966162" watchObservedRunningTime="2026-07-16 05:16:13.125335555 +0000 UTC m=+2225.850804078" Jul 16 05:16:22.210618 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:22.210566 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:22.210618 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:22.210632 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:22.223136 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:22.223103 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:23.175757 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:23.175722 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:16:44.023671 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:16:44.023622 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:17:22.119292 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:22.119258 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:17:22.220234 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:22.220200 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:17:25.359258 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:25.359222 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt"] Jul 16 05:17:25.382676 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:25.382649 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:17:26.303646 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:26.303616 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:17:30.413467 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:30.413431 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:17:30.511600 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:30.511567 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:17:40.379899 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.379852 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" podUID="a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3" containerName="main" containerID="cri-o://5085ff87aa7a8b2dd6b9250cc474a4cc4226d1c814683e88dbb231688343800c" gracePeriod=45 Jul 16 05:17:40.574318 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.574144 2581 generic.go:356] "Generic (PLEG): container finished" podID="a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3" containerID="5085ff87aa7a8b2dd6b9250cc474a4cc4226d1c814683e88dbb231688343800c" exitCode=0 Jul 16 05:17:40.574318 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.574230 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" event={"ID":"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3","Type":"ContainerDied","Data":"5085ff87aa7a8b2dd6b9250cc474a4cc4226d1c814683e88dbb231688343800c"} Jul 16 05:17:40.642146 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.642082 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:17:40.683164 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.683123 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptdk5\" (UniqueName: \"kubernetes.io/projected/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-kube-api-access-ptdk5\") pod \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " Jul 16 05:17:40.683164 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.683168 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-dshm\") pod \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " Jul 16 05:17:40.683433 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.683346 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-tls-certs\") pod \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " Jul 16 05:17:40.683433 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.683382 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-home\") pod \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " Jul 16 05:17:40.683575 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.683535 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-tmp-dir\") pod \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " Jul 16 05:17:40.683636 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.683578 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-model-cache\") pod \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\" (UID: \"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3\") " Jul 16 05:17:40.683880 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.683797 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3" (UID: "a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:17:40.683880 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.683809 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-home" (OuterVolumeSpecName: "home") pod "a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3" (UID: "a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:17:40.684061 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.683913 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-model-cache" (OuterVolumeSpecName: "model-cache") pod "a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3" (UID: "a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:17:40.684227 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.684208 2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-home\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:17:40.684274 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.684234 2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-tmp-dir\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:17:40.684274 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.684249 2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-model-cache\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:17:40.685652 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.685627 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3" (UID: "a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 05:17:40.685771 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.685731 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-dshm" (OuterVolumeSpecName: "dshm") pod "a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3" (UID: "a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:17:40.685858 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.685840 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-kube-api-access-ptdk5" (OuterVolumeSpecName: "kube-api-access-ptdk5") pod "a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3" (UID: "a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3"). InnerVolumeSpecName "kube-api-access-ptdk5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:17:40.785272 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.785229 2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-tls-certs\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:17:40.785272 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.785266 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptdk5\" (UniqueName: \"kubernetes.io/projected/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-kube-api-access-ptdk5\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:17:40.785272 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:40.785280 2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3-dshm\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:17:41.581190 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:41.581162 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" Jul 16 05:17:41.581658 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:41.581157 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt" event={"ID":"a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3","Type":"ContainerDied","Data":"2b016bf6fd54f7ddc82350c998d432012e329e180900da940d339d2c66c7f50e"} Jul 16 05:17:41.581658 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:41.581284 2581 scope.go:117] "RemoveContainer" containerID="5085ff87aa7a8b2dd6b9250cc474a4cc4226d1c814683e88dbb231688343800c" Jul 16 05:17:41.603162 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:41.603134 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt"] Jul 16 05:17:41.607172 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:41.607147 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a01a82/rate-test-model-a01a82-kserve-6ffc45948c-zk6qt"] Jul 16 05:17:41.840368 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:41.840286 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3" path="/var/lib/kubelet/pods/a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3/volumes" Jul 16 05:17:43.714875 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:43.714835 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:17:43.781440 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:43.781403 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:17:45.464933 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:45.464896 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:17:51.430841 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:51.430809 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:17:51.941585 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:51.941545 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:17:57.689537 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:57.689480 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:17:57.786580 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:57.786546 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:17:57.965707 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:57.965670 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89"] Jul 16 05:17:57.965952 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:57.965922 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" podUID="613495fb-b79a-4d1f-b786-e20f824cc7a6" containerName="istio-proxy" containerID="cri-o://a81296eaf06e846802bcb60d9f8aca356683ca0e39d617d25458ab167fdfc55e" gracePeriod=30 Jul 16 05:17:57.991472 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:57.991422 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:17:58.272562 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:58.272452 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:17:58.728780 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:17:58.728740 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:18:02.563252 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:02.563169 2581 patch_prober.go:28] interesting pod/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 05:18:02.563663 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:02.563263 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" podUID="613495fb-b79a-4d1f-b786-e20f824cc7a6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 05:18:03.225720 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.225655 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:18:03.309819 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.309779 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-token\") pod \"613495fb-b79a-4d1f-b786-e20f824cc7a6\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " Jul 16 05:18:03.310005 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.309857 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/613495fb-b79a-4d1f-b786-e20f824cc7a6-istiod-ca-cert\") pod \"613495fb-b79a-4d1f-b786-e20f824cc7a6\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " Jul 16 05:18:03.310005 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.309911 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-podinfo\") pod \"613495fb-b79a-4d1f-b786-e20f824cc7a6\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " Jul 16 05:18:03.310005 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.309939 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-workload-certs\") pod \"613495fb-b79a-4d1f-b786-e20f824cc7a6\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " Jul 16 05:18:03.310005 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.309983 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-workload-socket\") pod \"613495fb-b79a-4d1f-b786-e20f824cc7a6\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " Jul 16 05:18:03.310211 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.310021 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-envoy\") pod \"613495fb-b79a-4d1f-b786-e20f824cc7a6\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " Jul 16 05:18:03.310211 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.310049 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-data\") pod \"613495fb-b79a-4d1f-b786-e20f824cc7a6\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " Jul 16 05:18:03.310211 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.310084 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-credential-socket\") pod \"613495fb-b79a-4d1f-b786-e20f824cc7a6\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " Jul 16 05:18:03.310211 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.310107 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfl79\" (UniqueName: \"kubernetes.io/projected/613495fb-b79a-4d1f-b786-e20f824cc7a6-kube-api-access-cfl79\") pod \"613495fb-b79a-4d1f-b786-e20f824cc7a6\" (UID: \"613495fb-b79a-4d1f-b786-e20f824cc7a6\") " Jul 16 05:18:03.310414 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.310328 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "613495fb-b79a-4d1f-b786-e20f824cc7a6" (UID: "613495fb-b79a-4d1f-b786-e20f824cc7a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:18:03.310475 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.310346 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/613495fb-b79a-4d1f-b786-e20f824cc7a6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "613495fb-b79a-4d1f-b786-e20f824cc7a6" (UID: "613495fb-b79a-4d1f-b786-e20f824cc7a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:18:03.310475 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.310422 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "613495fb-b79a-4d1f-b786-e20f824cc7a6" (UID: "613495fb-b79a-4d1f-b786-e20f824cc7a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:18:03.310622 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.310566 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/613495fb-b79a-4d1f-b786-e20f824cc7a6-istiod-ca-cert\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:18:03.310622 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.310585 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-workload-certs\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:18:03.310622 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.310598 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-workload-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:18:03.310805 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.310787 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "613495fb-b79a-4d1f-b786-e20f824cc7a6" (UID: "613495fb-b79a-4d1f-b786-e20f824cc7a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:18:03.311077 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.311053 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-data" (OuterVolumeSpecName: "istio-data") pod "613495fb-b79a-4d1f-b786-e20f824cc7a6" (UID: "613495fb-b79a-4d1f-b786-e20f824cc7a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:18:03.312258 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.312228 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-token" (OuterVolumeSpecName: "istio-token") pod "613495fb-b79a-4d1f-b786-e20f824cc7a6" (UID: "613495fb-b79a-4d1f-b786-e20f824cc7a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:18:03.312464 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.312444 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/613495fb-b79a-4d1f-b786-e20f824cc7a6-kube-api-access-cfl79" (OuterVolumeSpecName: "kube-api-access-cfl79") pod "613495fb-b79a-4d1f-b786-e20f824cc7a6" (UID: "613495fb-b79a-4d1f-b786-e20f824cc7a6"). InnerVolumeSpecName "kube-api-access-cfl79". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:18:03.312594 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.312569 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "613495fb-b79a-4d1f-b786-e20f824cc7a6" (UID: "613495fb-b79a-4d1f-b786-e20f824cc7a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:18:03.312649 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.312580 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "613495fb-b79a-4d1f-b786-e20f824cc7a6" (UID: "613495fb-b79a-4d1f-b786-e20f824cc7a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:18:03.411879 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.411837 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-data\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:18:03.411879 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.411871 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-credential-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:18:03.411879 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.411884 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfl79\" (UniqueName: \"kubernetes.io/projected/613495fb-b79a-4d1f-b786-e20f824cc7a6-kube-api-access-cfl79\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:18:03.412130 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.411896 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-token\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:18:03.412130 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.411908 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-podinfo\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:18:03.412130 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.411918 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/613495fb-b79a-4d1f-b786-e20f824cc7a6-istio-envoy\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:18:03.699375 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.699281 2581 generic.go:356] "Generic (PLEG): container finished" podID="613495fb-b79a-4d1f-b786-e20f824cc7a6" containerID="a81296eaf06e846802bcb60d9f8aca356683ca0e39d617d25458ab167fdfc55e" exitCode=0 Jul 16 05:18:03.699375 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.699332 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" Jul 16 05:18:03.699897 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.699372 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" event={"ID":"613495fb-b79a-4d1f-b786-e20f824cc7a6","Type":"ContainerDied","Data":"a81296eaf06e846802bcb60d9f8aca356683ca0e39d617d25458ab167fdfc55e"} Jul 16 05:18:03.699897 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.699435 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89" event={"ID":"613495fb-b79a-4d1f-b786-e20f824cc7a6","Type":"ContainerDied","Data":"e6bb446d6727454e20091950dc8e25a9ebe69798b5afdb28dd26134a4ac40bc5"} Jul 16 05:18:03.699897 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.699468 2581 scope.go:117] "RemoveContainer" containerID="a81296eaf06e846802bcb60d9f8aca356683ca0e39d617d25458ab167fdfc55e" Jul 16 05:18:03.708859 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.708840 2581 scope.go:117] "RemoveContainer" containerID="9629c1a4d9cc91856d95646c7970a2a43e72443b84b26285914d40ecc2cd4ab3" Jul 16 05:18:03.722866 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.722838 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89"] Jul 16 05:18:03.723063 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.723020 2581 scope.go:117] "RemoveContainer" containerID="a81296eaf06e846802bcb60d9f8aca356683ca0e39d617d25458ab167fdfc55e" Jul 16 05:18:03.723313 ip-10-0-132-245 kubenswrapper[2581]: E0716 05:18:03.723292 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a81296eaf06e846802bcb60d9f8aca356683ca0e39d617d25458ab167fdfc55e\": container with ID starting with a81296eaf06e846802bcb60d9f8aca356683ca0e39d617d25458ab167fdfc55e not found: ID does not exist" containerID="a81296eaf06e846802bcb60d9f8aca356683ca0e39d617d25458ab167fdfc55e" Jul 16 05:18:03.723373 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.723320 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a81296eaf06e846802bcb60d9f8aca356683ca0e39d617d25458ab167fdfc55e"} err="failed to get container status \"a81296eaf06e846802bcb60d9f8aca356683ca0e39d617d25458ab167fdfc55e\": rpc error: code = NotFound desc = could not find container \"a81296eaf06e846802bcb60d9f8aca356683ca0e39d617d25458ab167fdfc55e\": container with ID starting with a81296eaf06e846802bcb60d9f8aca356683ca0e39d617d25458ab167fdfc55e not found: ID does not exist" Jul 16 05:18:03.723373 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.723338 2581 scope.go:117] "RemoveContainer" containerID="9629c1a4d9cc91856d95646c7970a2a43e72443b84b26285914d40ecc2cd4ab3" Jul 16 05:18:03.723596 ip-10-0-132-245 kubenswrapper[2581]: E0716 05:18:03.723580 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9629c1a4d9cc91856d95646c7970a2a43e72443b84b26285914d40ecc2cd4ab3\": container with ID starting with 9629c1a4d9cc91856d95646c7970a2a43e72443b84b26285914d40ecc2cd4ab3 not found: ID does not exist" containerID="9629c1a4d9cc91856d95646c7970a2a43e72443b84b26285914d40ecc2cd4ab3" Jul 16 05:18:03.723648 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.723600 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9629c1a4d9cc91856d95646c7970a2a43e72443b84b26285914d40ecc2cd4ab3"} err="failed to get container status \"9629c1a4d9cc91856d95646c7970a2a43e72443b84b26285914d40ecc2cd4ab3\": rpc error: code = NotFound desc = could not find container \"9629c1a4d9cc91856d95646c7970a2a43e72443b84b26285914d40ecc2cd4ab3\": container with ID starting with 9629c1a4d9cc91856d95646c7970a2a43e72443b84b26285914d40ecc2cd4ab3 not found: ID does not exist" Jul 16 05:18:03.726990 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.726966 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a01a82-openshift-default-5d9c54f75f-rvh89"] Jul 16 05:18:03.839871 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:03.839836 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="613495fb-b79a-4d1f-b786-e20f824cc7a6" path="/var/lib/kubelet/pods/613495fb-b79a-4d1f-b786-e20f824cc7a6/volumes" Jul 16 05:18:04.227922 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:04.227874 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:18:04.634760 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:04.634674 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:18:10.344777 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:10.344742 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:18:10.424099 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:10.424058 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:18:10.582966 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:10.582925 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498"] Jul 16 05:18:10.583271 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:10.583232 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" podUID="7ab1851e-c352-470b-9278-3136d221f2be" containerName="istio-proxy" containerID="cri-o://06223757d328576c51c77b029d89d9af97970147831d684ea9cdee7531fe8038" gracePeriod=30 Jul 16 05:18:10.599766 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:10.599682 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:18:10.857167 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:10.857089 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:18:11.308164 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:11.308116 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56cfd66f6d-9pxxq_c069473c-638f-472f-aba7-f32ed57adc6e/authorino/0.log" Jul 16 05:18:14.297337 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:14.297303 2581 patch_prober.go:28] interesting pod/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 05:18:14.297780 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:14.297383 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" podUID="7ab1851e-c352-470b-9278-3136d221f2be" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 05:18:15.764096 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.764067 2581 generic.go:356] "Generic (PLEG): container finished" podID="7ab1851e-c352-470b-9278-3136d221f2be" containerID="06223757d328576c51c77b029d89d9af97970147831d684ea9cdee7531fe8038" exitCode=0 Jul 16 05:18:15.764449 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.764143 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" event={"ID":"7ab1851e-c352-470b-9278-3136d221f2be","Type":"ContainerDied","Data":"06223757d328576c51c77b029d89d9af97970147831d684ea9cdee7531fe8038"} Jul 16 05:18:15.764449 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.764217 2581 scope.go:117] "RemoveContainer" containerID="24943588ff5777288afc5093922ba0d0a318e7b16d2d2847bd3e085900b04f8d" Jul 16 05:18:15.847056 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.846994 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:18:15.927181 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.927141 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ab1851e-c352-470b-9278-3136d221f2be-istiod-ca-cert\") pod \"7ab1851e-c352-470b-9278-3136d221f2be\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " Jul 16 05:18:15.927417 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.927193 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ab1851e-c352-470b-9278-3136d221f2be-istio-token\") pod \"7ab1851e-c352-470b-9278-3136d221f2be\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " Jul 16 05:18:15.927417 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.927225 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ab1851e-c352-470b-9278-3136d221f2be-istio-podinfo\") pod \"7ab1851e-c352-470b-9278-3136d221f2be\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " Jul 16 05:18:15.927417 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.927273 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-workload-certs\") pod \"7ab1851e-c352-470b-9278-3136d221f2be\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " Jul 16 05:18:15.927417 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.927305 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-istio-envoy\") pod \"7ab1851e-c352-470b-9278-3136d221f2be\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " Jul 16 05:18:15.927417 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.927328 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4pbr\" (UniqueName: \"kubernetes.io/projected/7ab1851e-c352-470b-9278-3136d221f2be-kube-api-access-f4pbr\") pod \"7ab1851e-c352-470b-9278-3136d221f2be\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " Jul 16 05:18:15.927417 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.927365 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-istio-data\") pod \"7ab1851e-c352-470b-9278-3136d221f2be\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " Jul 16 05:18:15.927417 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.927412 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-workload-socket\") pod \"7ab1851e-c352-470b-9278-3136d221f2be\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " Jul 16 05:18:15.927813 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.927454 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-credential-socket\") pod \"7ab1851e-c352-470b-9278-3136d221f2be\" (UID: \"7ab1851e-c352-470b-9278-3136d221f2be\") " Jul 16 05:18:15.927881 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.927837 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7ab1851e-c352-470b-9278-3136d221f2be" (UID: "7ab1851e-c352-470b-9278-3136d221f2be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:18:15.928062 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.927947 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ab1851e-c352-470b-9278-3136d221f2be-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7ab1851e-c352-470b-9278-3136d221f2be" (UID: "7ab1851e-c352-470b-9278-3136d221f2be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:18:15.928162 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.928024 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7ab1851e-c352-470b-9278-3136d221f2be" (UID: "7ab1851e-c352-470b-9278-3136d221f2be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:18:15.928162 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.928071 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7ab1851e-c352-470b-9278-3136d221f2be" (UID: "7ab1851e-c352-470b-9278-3136d221f2be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:18:15.928405 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.928377 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-istio-data" (OuterVolumeSpecName: "istio-data") pod "7ab1851e-c352-470b-9278-3136d221f2be" (UID: "7ab1851e-c352-470b-9278-3136d221f2be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:18:15.929694 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.929664 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ab1851e-c352-470b-9278-3136d221f2be-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7ab1851e-c352-470b-9278-3136d221f2be" (UID: "7ab1851e-c352-470b-9278-3136d221f2be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:18:15.929811 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.929695 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ab1851e-c352-470b-9278-3136d221f2be-istio-token" (OuterVolumeSpecName: "istio-token") pod "7ab1851e-c352-470b-9278-3136d221f2be" (UID: "7ab1851e-c352-470b-9278-3136d221f2be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:18:15.929811 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.929766 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7ab1851e-c352-470b-9278-3136d221f2be" (UID: "7ab1851e-c352-470b-9278-3136d221f2be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:18:15.929895 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:15.929851 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ab1851e-c352-470b-9278-3136d221f2be-kube-api-access-f4pbr" (OuterVolumeSpecName: "kube-api-access-f4pbr") pod "7ab1851e-c352-470b-9278-3136d221f2be" (UID: "7ab1851e-c352-470b-9278-3136d221f2be"). InnerVolumeSpecName "kube-api-access-f4pbr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:18:16.028591 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:16.028556 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-istio-envoy\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:18:16.028591 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:16.028584 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4pbr\" (UniqueName: \"kubernetes.io/projected/7ab1851e-c352-470b-9278-3136d221f2be-kube-api-access-f4pbr\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:18:16.028591 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:16.028597 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-istio-data\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:18:16.028829 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:16.028605 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-workload-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:18:16.028829 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:16.028613 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-credential-socket\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:18:16.028829 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:16.028622 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ab1851e-c352-470b-9278-3136d221f2be-istiod-ca-cert\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:18:16.028829 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:16.028630 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ab1851e-c352-470b-9278-3136d221f2be-istio-token\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:18:16.028829 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:16.028637 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ab1851e-c352-470b-9278-3136d221f2be-istio-podinfo\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:18:16.028829 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:16.028644 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ab1851e-c352-470b-9278-3136d221f2be-workload-certs\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\"" Jul 16 05:18:16.771970 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:16.771936 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" Jul 16 05:18:16.772450 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:16.771987 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498" event={"ID":"7ab1851e-c352-470b-9278-3136d221f2be","Type":"ContainerDied","Data":"ab09ddef4ecf15599361dce68c43232fdd3e89fe01411e255fcb6b45cd7344cf"} Jul 16 05:18:16.772450 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:16.772053 2581 scope.go:117] "RemoveContainer" containerID="06223757d328576c51c77b029d89d9af97970147831d684ea9cdee7531fe8038" Jul 16 05:18:16.793076 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:16.793045 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498"] Jul 16 05:18:16.796737 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:16.796715 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-12efc5-openshift-default-d64bfcdc-26498"] Jul 16 05:18:17.023711 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:17.023633 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-w7tqh_1c73a1b4-1f36-45bb-a3f0-882ee8ef4a50/manager/0.log" Jul 16 05:18:17.161522 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:17.161467 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7hpfm_e438ea2c-9d44-4919-b90f-6f7ef2d86612/manager/0.log" Jul 16 05:18:17.287279 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:17.287198 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5b6d84dc7c-kljmf_b261b326-c0c6-4bb6-8122-3a91bb8301ab/manager/0.log" Jul 16 05:18:17.404436 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:17.404410 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-xgwlw_39f2d9d3-4b27-4b65-9725-7b30babbe5e5/server/0.log" Jul 16 05:18:17.532644 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:17.532613 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2q482_4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5/manager/0.log" Jul 16 05:18:17.790889 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:17.790854 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b4b7cb559-l262f_48e126e9-56a9-43eb-b98b-b6494445cbeb/copy-manifests/0.log" Jul 16 05:18:17.803191 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:17.803164 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b4b7cb559-l262f_48e126e9-56a9-43eb-b98b-b6494445cbeb/manager/0.log" Jul 16 05:18:17.840882 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:17.840849 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ab1851e-c352-470b-9278-3136d221f2be" path="/var/lib/kubelet/pods/7ab1851e-c352-470b-9278-3136d221f2be/volumes" Jul 16 05:18:18.295347 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:18.295318 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-76569bc6fb-p9sjm_89cea744-972e-4854-bfcc-19575fb43e30/maas-api/0.log" Jul 16 05:18:18.652291 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:18.652215 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-c9c47_af0cf299-db3c-4a03-b0c4-33e36c883a33/postgres/0.log" Jul 16 05:18:20.056995 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:20.056963 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56cfd66f6d-9pxxq_c069473c-638f-472f-aba7-f32ed57adc6e/authorino/0.log" Jul 16 05:18:20.207416 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:20.207388 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-2f5mn_bd850487-c720-46c5-a704-7d60238b0e6b/manager/0.log" Jul 16 05:18:20.326007 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:20.325926 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-s8f5d_2b837d07-e062-4906-93b3-eec0ef7b091b/manager/0.log" Jul 16 05:18:20.454011 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:20.453983 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-7qr9d_c381c339-c947-4130-8568-d2eca3f56ef6/kuadrant-console-plugin/0.log" Jul 16 05:18:20.586573 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:20.586465 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-tvvv9_f99d39f4-5750-4fab-bef8-a8cd7f2e9e6f/registry-server/0.log" Jul 16 05:18:21.000386 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:21.000352 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-8f88k_d66f64fc-f7c0-4b56-befd-6fad4182d62e/manager/0.log" Jul 16 05:18:21.375056 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:21.374979 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nhbqk_89b954af-2c48-4a5e-a06c-6c0bb14cfc69/istio-proxy/0.log" Jul 16 05:18:21.723092 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:21.723064 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68764b5749-fddrb_9351e506-d7ac-4be8-a5dc-84f1d25515ef/kube-auth-proxy/0.log" Jul 16 05:18:21.843131 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:21.843106 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-llv9g_9c241788-0107-4d03-a926-7a4bf3c806db/istio-proxy/1.log" Jul 16 05:18:22.095995 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:22.095909 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-6cqdj_bbf6a1d3-485d-4840-9920-33d23f331880/payload-processing/0.log" Jul 16 05:18:22.697137 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:22.697108 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-bwrcl_7979c811-3f46-4b9b-8367-725521e261dc/main/0.log" Jul 16 05:18:23.049865 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:23.049835 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-gqnlk_fe086561-4ba6-410b-a3af-2d0dd032f572/main/0.log" Jul 16 05:18:28.869705 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:28.869673 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:18:28.992158 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:28.992126 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" limit=1024 Jul 16 05:18:31.863441 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:31.863407 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-g6vzk_5da9fe69-0b36-4198-a5ff-329ddaee7389/global-pull-secret-syncer/0.log" Jul 16 05:18:32.014943 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:32.014908 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fnkvt_3b736bea-a20b-4220-b592-3819366c1ebd/konnectivity-agent/0.log" Jul 16 05:18:32.136699 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:32.136605 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-245.ec2.internal_40de15c40dab007c9ee75dd5f90e7e7c/haproxy/0.log" Jul 16 05:18:37.399733 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:37.399695 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56cfd66f6d-9pxxq_c069473c-638f-472f-aba7-f32ed57adc6e/authorino/0.log" Jul 16 05:18:37.426037 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:37.426009 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-2f5mn_bd850487-c720-46c5-a704-7d60238b0e6b/manager/0.log" Jul 16 05:18:37.458884 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:37.458855 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-s8f5d_2b837d07-e062-4906-93b3-eec0ef7b091b/manager/0.log" Jul 16 05:18:37.486188 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:37.486151 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-7qr9d_c381c339-c947-4130-8568-d2eca3f56ef6/kuadrant-console-plugin/0.log" Jul 16 05:18:37.526796 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:37.526767 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-tvvv9_f99d39f4-5750-4fab-bef8-a8cd7f2e9e6f/registry-server/0.log" Jul 16 05:18:37.737539 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:37.737490 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-8f88k_d66f64fc-f7c0-4b56-befd-6fad4182d62e/manager/0.log" Jul 16 05:18:39.640133 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:39.640090 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jkmvx_b1c7ae8c-3aca-4b72-827d-672b3d14dea6/node-exporter/0.log" Jul 16 05:18:39.663282 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:39.663252 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jkmvx_b1c7ae8c-3aca-4b72-827d-672b3d14dea6/kube-rbac-proxy/0.log" Jul 16 05:18:39.682935 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:39.682910 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jkmvx_b1c7ae8c-3aca-4b72-827d-672b3d14dea6/init-textfile/0.log" Jul 16 05:18:40.215349 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:40.215315 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-659f84fb8-fvjgx_cc2ce8d2-bc2d-4380-a3e6-53532c708b62/thanos-query/0.log" Jul 16 05:18:40.237408 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:40.237361 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-659f84fb8-fvjgx_cc2ce8d2-bc2d-4380-a3e6-53532c708b62/kube-rbac-proxy-web/0.log" Jul 16 05:18:40.263864 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:40.263835 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-659f84fb8-fvjgx_cc2ce8d2-bc2d-4380-a3e6-53532c708b62/kube-rbac-proxy/0.log" Jul 16 05:18:40.285232 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:40.285208 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-659f84fb8-fvjgx_cc2ce8d2-bc2d-4380-a3e6-53532c708b62/prom-label-proxy/0.log" Jul 16 05:18:40.317156 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:40.317129 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-659f84fb8-fvjgx_cc2ce8d2-bc2d-4380-a3e6-53532c708b62/kube-rbac-proxy-rules/0.log" Jul 16 05:18:40.342229 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:40.342204 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-659f84fb8-fvjgx_cc2ce8d2-bc2d-4380-a3e6-53532c708b62/kube-rbac-proxy-metrics/0.log" Jul 16 05:18:41.560271 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:41.560242 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-6zfcn_566deb82-6b60-4c93-aaeb-556a2e9bd270/networking-console-plugin/0.log" Jul 16 05:18:41.992323 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:41.992290 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg"] Jul 16 05:18:41.992742 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:41.992727 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="613495fb-b79a-4d1f-b786-e20f824cc7a6" containerName="istio-proxy" Jul 16 05:18:41.992804 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:41.992745 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="613495fb-b79a-4d1f-b786-e20f824cc7a6" containerName="istio-proxy" Jul 16 05:18:41.992804 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:41.992765 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3" containerName="main" Jul 16 05:18:41.992804 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:41.992772 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3" containerName="main" Jul 16 05:18:41.992804 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:41.992791 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ab1851e-c352-470b-9278-3136d221f2be" containerName="istio-proxy" Jul 16 05:18:41.992804 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:41.992797 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ab1851e-c352-470b-9278-3136d221f2be" containerName="istio-proxy" Jul 16 05:18:41.992804 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:41.992804 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ab1851e-c352-470b-9278-3136d221f2be" containerName="istio-proxy" Jul 16 05:18:41.992986 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:41.992809 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ab1851e-c352-470b-9278-3136d221f2be" containerName="istio-proxy" Jul 16 05:18:41.992986 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:41.992878 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="613495fb-b79a-4d1f-b786-e20f824cc7a6" containerName="istio-proxy" Jul 16 05:18:41.992986 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:41.992888 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="613495fb-b79a-4d1f-b786-e20f824cc7a6" containerName="istio-proxy" Jul 16 05:18:41.992986 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:41.992897 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ab1851e-c352-470b-9278-3136d221f2be" containerName="istio-proxy" Jul 16 05:18:41.992986 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:41.992904 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ab1851e-c352-470b-9278-3136d221f2be" containerName="istio-proxy" Jul 16 05:18:41.992986 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:41.992912 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6d573c3-79e8-4b1c-9c14-cfbbaf1f6dc3" containerName="main" Jul 16 05:18:41.997748 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:41.997727 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:42.000271 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.000244 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5c2z4\"/\"kube-root-ca.crt\"" Jul 16 05:18:42.000385 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.000292 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5c2z4\"/\"default-dockercfg-jr99c\"" Jul 16 05:18:42.000446 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.000389 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5c2z4\"/\"openshift-service-ca.crt\"" Jul 16 05:18:42.005931 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.005906 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg"] Jul 16 05:18:42.068121 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.068087 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18f6cdf7-5c7a-45c8-b80d-6614d0b5585d-lib-modules\") pod \"perf-node-gather-daemonset-w7vlg\" (UID: \"18f6cdf7-5c7a-45c8-b80d-6614d0b5585d\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:42.068302 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.068129 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18f6cdf7-5c7a-45c8-b80d-6614d0b5585d-proc\") pod \"perf-node-gather-daemonset-w7vlg\" (UID: \"18f6cdf7-5c7a-45c8-b80d-6614d0b5585d\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:42.068302 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.068166 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18f6cdf7-5c7a-45c8-b80d-6614d0b5585d-podres\") pod \"perf-node-gather-daemonset-w7vlg\" (UID: \"18f6cdf7-5c7a-45c8-b80d-6614d0b5585d\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:42.068302 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.068238 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18f6cdf7-5c7a-45c8-b80d-6614d0b5585d-sys\") pod \"perf-node-gather-daemonset-w7vlg\" (UID: \"18f6cdf7-5c7a-45c8-b80d-6614d0b5585d\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:42.068432 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.068319 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9p9n\" (UniqueName: \"kubernetes.io/projected/18f6cdf7-5c7a-45c8-b80d-6614d0b5585d-kube-api-access-n9p9n\") pod \"perf-node-gather-daemonset-w7vlg\" (UID: \"18f6cdf7-5c7a-45c8-b80d-6614d0b5585d\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:42.169297 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.169252 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9p9n\" (UniqueName: \"kubernetes.io/projected/18f6cdf7-5c7a-45c8-b80d-6614d0b5585d-kube-api-access-n9p9n\") pod \"perf-node-gather-daemonset-w7vlg\" (UID: \"18f6cdf7-5c7a-45c8-b80d-6614d0b5585d\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:42.169539 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.169322 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18f6cdf7-5c7a-45c8-b80d-6614d0b5585d-lib-modules\") pod \"perf-node-gather-daemonset-w7vlg\" (UID: \"18f6cdf7-5c7a-45c8-b80d-6614d0b5585d\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:42.169539 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.169347 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18f6cdf7-5c7a-45c8-b80d-6614d0b5585d-proc\") pod \"perf-node-gather-daemonset-w7vlg\" (UID: \"18f6cdf7-5c7a-45c8-b80d-6614d0b5585d\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:42.169539 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.169383 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18f6cdf7-5c7a-45c8-b80d-6614d0b5585d-podres\") pod \"perf-node-gather-daemonset-w7vlg\" (UID: \"18f6cdf7-5c7a-45c8-b80d-6614d0b5585d\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:42.169539 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.169401 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18f6cdf7-5c7a-45c8-b80d-6614d0b5585d-sys\") pod \"perf-node-gather-daemonset-w7vlg\" (UID: \"18f6cdf7-5c7a-45c8-b80d-6614d0b5585d\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:42.169539 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.169481 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18f6cdf7-5c7a-45c8-b80d-6614d0b5585d-sys\") pod \"perf-node-gather-daemonset-w7vlg\" (UID: \"18f6cdf7-5c7a-45c8-b80d-6614d0b5585d\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:42.169539 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.169479 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18f6cdf7-5c7a-45c8-b80d-6614d0b5585d-proc\") pod \"perf-node-gather-daemonset-w7vlg\" (UID: \"18f6cdf7-5c7a-45c8-b80d-6614d0b5585d\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:42.169780 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.169551 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18f6cdf7-5c7a-45c8-b80d-6614d0b5585d-podres\") pod \"perf-node-gather-daemonset-w7vlg\" (UID: \"18f6cdf7-5c7a-45c8-b80d-6614d0b5585d\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:42.169780 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.169546 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18f6cdf7-5c7a-45c8-b80d-6614d0b5585d-lib-modules\") pod \"perf-node-gather-daemonset-w7vlg\" (UID: \"18f6cdf7-5c7a-45c8-b80d-6614d0b5585d\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:42.177581 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.177553 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9p9n\" (UniqueName: \"kubernetes.io/projected/18f6cdf7-5c7a-45c8-b80d-6614d0b5585d-kube-api-access-n9p9n\") pod \"perf-node-gather-daemonset-w7vlg\" (UID: \"18f6cdf7-5c7a-45c8-b80d-6614d0b5585d\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:42.309416 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.309311 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:42.448352 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.448326 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg"] Jul 16 05:18:42.450320 ip-10-0-132-245 kubenswrapper[2581]: W0716 05:18:42.450294 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod18f6cdf7_5c7a_45c8_b80d_6614d0b5585d.slice/crio-a89d101ed5949166062c8c16b2630bec39ed5721bbce924985b85e461e6e3992 WatchSource:0}: Error finding container a89d101ed5949166062c8c16b2630bec39ed5721bbce924985b85e461e6e3992: Status 404 returned error can't find the container with id a89d101ed5949166062c8c16b2630bec39ed5721bbce924985b85e461e6e3992 Jul 16 05:18:42.907823 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.907788 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" event={"ID":"18f6cdf7-5c7a-45c8-b80d-6614d0b5585d","Type":"ContainerStarted","Data":"58d3b0a838823ec5f121128fb6e7aa427f3030d904083c88e954f7b5ea6b6658"} Jul 16 05:18:42.908244 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.907851 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" event={"ID":"18f6cdf7-5c7a-45c8-b80d-6614d0b5585d","Type":"ContainerStarted","Data":"a89d101ed5949166062c8c16b2630bec39ed5721bbce924985b85e461e6e3992"} Jul 16 05:18:42.908244 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:42.907936 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:43.566935 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:43.566908 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-v96fw_4b84faae-b7d2-45c5-b73e-4f597522ea49/volume-data-source-validator/0.log" Jul 16 05:18:44.690196 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:44.690165 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hjvdt_6419d78a-657f-4e8c-8ea4-6f5a4f92c234/dns/0.log" Jul 16 05:18:44.711940 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:44.711915 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hjvdt_6419d78a-657f-4e8c-8ea4-6f5a4f92c234/kube-rbac-proxy/0.log" Jul 16 05:18:44.827662 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:44.827635 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xjxgr_01e9bde7-f187-4fe4-b933-e31dfc8de92b/dns-node-resolver/0.log" Jul 16 05:18:45.461756 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:45.461724 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cngtn_852b5867-4340-4c84-9da8-8e4f02270f0d/node-ca/0.log" Jul 16 05:18:46.458010 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:46.457979 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nhbqk_89b954af-2c48-4a5e-a06c-6c0bb14cfc69/istio-proxy/0.log" Jul 16 05:18:46.611776 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:46.611747 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68764b5749-fddrb_9351e506-d7ac-4be8-a5dc-84f1d25515ef/kube-auth-proxy/0.log" Jul 16 05:18:46.644346 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:46.644312 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-llv9g_9c241788-0107-4d03-a926-7a4bf3c806db/istio-proxy/0.log" Jul 16 05:18:46.673785 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:46.673749 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-llv9g_9c241788-0107-4d03-a926-7a4bf3c806db/istio-proxy/1.log" Jul 16 05:18:46.779658 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:46.779576 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-6cqdj_bbf6a1d3-485d-4840-9920-33d23f331880/payload-processing/0.log" Jul 16 05:18:47.495327 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:47.495296 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kdmwr_2f60f16a-e6e4-4289-b19b-18c9c3826d21/serve-healthcheck-canary/0.log" Jul 16 05:18:48.029211 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:48.029174 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-cgw2x_aaa2dfa0-68ee-427d-86dc-bb788a81aec5/insights-operator/0.log" Jul 16 05:18:48.029630 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:48.029614 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-cgw2x_aaa2dfa0-68ee-427d-86dc-bb788a81aec5/insights-operator/1.log" Jul 16 05:18:48.193040 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:48.193005 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w92t7_794de458-b940-478f-a21b-9d2450761e3d/kube-rbac-proxy/0.log" Jul 16 05:18:48.214134 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:48.214103 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w92t7_794de458-b940-478f-a21b-9d2450761e3d/exporter/0.log" Jul 16 05:18:48.236852 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:48.236825 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w92t7_794de458-b940-478f-a21b-9d2450761e3d/extractor/0.log" Jul 16 05:18:48.923311 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:48.923277 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" Jul 16 05:18:48.941171 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:48.941105 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-w7vlg" podStartSLOduration=7.941089079 podStartE2EDuration="7.941089079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 05:18:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:18:42.92406906 +0000 UTC m=+2375.649537582" watchObservedRunningTime="2026-07-16 05:18:48.941089079 +0000 UTC m=+2381.666557602" Jul 16 05:18:50.407245 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:50.407167 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-w7tqh_1c73a1b4-1f36-45bb-a3f0-882ee8ef4a50/manager/0.log" Jul 16 05:18:50.535516 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:50.535483 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7hpfm_e438ea2c-9d44-4919-b90f-6f7ef2d86612/manager/0.log" Jul 16 05:18:50.642911 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:50.642875 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5b6d84dc7c-kljmf_b261b326-c0c6-4bb6-8122-3a91bb8301ab/manager/0.log" Jul 16 05:18:50.663262 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:50.663189 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-xgwlw_39f2d9d3-4b27-4b65-9725-7b30babbe5e5/server/0.log" Jul 16 05:18:50.724938 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:50.724905 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2q482_4ee5c1cd-0c5a-4cf7-b61f-e87d3f687ab5/manager/0.log" Jul 16 05:18:50.851479 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:50.851439 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b4b7cb559-l262f_48e126e9-56a9-43eb-b98b-b6494445cbeb/manager/0.log" Jul 16 05:18:50.870886 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:50.870862 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b4b7cb559-l262f_48e126e9-56a9-43eb-b98b-b6494445cbeb/copy-manifests/0.log" Jul 16 05:18:52.372834 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:52.372752 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-6dhq2_2516dbd5-08da-4e57-b9f5-c3bab3f1c50e/openshift-lws-operator/0.log" Jul 16 05:18:52.637890 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:52.637804 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nhbqk_89b954af-2c48-4a5e-a06c-6c0bb14cfc69/istio-proxy/0.log" Jul 16 05:18:52.766246 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:52.766208 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68764b5749-fddrb_9351e506-d7ac-4be8-a5dc-84f1d25515ef/kube-auth-proxy/0.log" Jul 16 05:18:52.785492 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:52.785460 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-llv9g_9c241788-0107-4d03-a926-7a4bf3c806db/istio-proxy/0.log" Jul 16 05:18:52.815446 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:52.815409 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-llv9g_9c241788-0107-4d03-a926-7a4bf3c806db/istio-proxy/1.log" Jul 16 05:18:52.865739 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:52.865691 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-6cqdj_bbf6a1d3-485d-4840-9920-33d23f331880/payload-processing/0.log" Jul 16 05:18:54.114534 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:54.114415 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nhbqk_89b954af-2c48-4a5e-a06c-6c0bb14cfc69/istio-proxy/0.log" Jul 16 05:18:54.244370 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:54.244339 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68764b5749-fddrb_9351e506-d7ac-4be8-a5dc-84f1d25515ef/kube-auth-proxy/0.log" Jul 16 05:18:54.262324 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:54.262289 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-llv9g_9c241788-0107-4d03-a926-7a4bf3c806db/istio-proxy/0.log" Jul 16 05:18:54.290991 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:54.290944 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-llv9g_9c241788-0107-4d03-a926-7a4bf3c806db/istio-proxy/1.log" Jul 16 05:18:54.347564 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:54.347538 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-6cqdj_bbf6a1d3-485d-4840-9920-33d23f331880/payload-processing/0.log" Jul 16 05:18:57.773649 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:57.773614 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-6vqq8_fb7574c6-5d40-40ed-b909-0abde38d01ba/kube-storage-version-migrator-operator/1.log" Jul 16 05:18:57.774437 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:57.774409 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-6vqq8_fb7574c6-5d40-40ed-b909-0abde38d01ba/kube-storage-version-migrator-operator/0.log" Jul 16 05:18:59.086861 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:59.086832 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m9m2p_99e2e193-30b8-43ed-9e81-023f10d73b10/kube-multus-additional-cni-plugins/0.log" Jul 16 05:18:59.107641 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:59.107605 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m9m2p_99e2e193-30b8-43ed-9e81-023f10d73b10/egress-router-binary-copy/0.log" Jul 16 05:18:59.128373 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:59.128346 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m9m2p_99e2e193-30b8-43ed-9e81-023f10d73b10/cni-plugins/0.log" Jul 16 05:18:59.149363 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:59.149332 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m9m2p_99e2e193-30b8-43ed-9e81-023f10d73b10/bond-cni-plugin/0.log" Jul 16 05:18:59.169591 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:59.169560 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m9m2p_99e2e193-30b8-43ed-9e81-023f10d73b10/routeoverride-cni/0.log" Jul 16 05:18:59.190321 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:59.190291 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m9m2p_99e2e193-30b8-43ed-9e81-023f10d73b10/whereabouts-cni-bincopy/0.log" Jul 16 05:18:59.211782 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:59.211753 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m9m2p_99e2e193-30b8-43ed-9e81-023f10d73b10/whereabouts-cni/0.log" Jul 16 05:18:59.244158 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:59.244128 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-hnbvz_0d695a61-7e54-42ac-9251-a3d84eeb3baa/kube-multus/0.log" Jul 16 05:18:59.310837 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:59.310804 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7g5n5_2f51d802-9618-4e88-8fc8-78b2a3788356/network-metrics-daemon/0.log" Jul 16 05:18:59.330517 ip-10-0-132-245 kubenswrapper[2581]: I0716 05:18:59.330488 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7g5n5_2f51d802-9618-4e88-8fc8-78b2a3788356/kube-rbac-proxy/0.log"